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I. INTRODUCTION 



In the past twenty years, the computer industry has 
sustained a technical reevaluation unrivaled in modern 
history. The computer has become the greatest management 
tool of our time.j Yet its proper application contains many 
pitfalls, as case after case of dramatic failures testify. 

One of these dangers is improper equipment selection. When 
managers thoughtlessly procure equipment as a natural process 
item, they can easily preclude any possibility of success 
simply by buying the wrong equipment. 

The computer selection and evaluation process has become 
a complex one, requiring detailed attention; it can involve 
hundreds of technical as well as nontechnical considerations. 
Today's computer systems are typically very complex and 
extensive in composition and operation. Academic and admin- 
istrative users of computer systems have traditionally left 
most of the considerations in systems selection to technical 
personnel. As a result, many user needs have gone unsatis- 
fied. Technicians have become frustrated because they often 
found out too late, if at all, what user needs and priorities 
really were. In addition, many technicians have had diffi- 
culty in communicating to relatively untrained users a 
complete understanding of the actual capabilities of various 
computer systems. 

The remarkable technical reevaluation in the industry has 
led to the creation and ultimate availability of a large 
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number of unique computer systems. When one considers the 
vast number of peripheral devices available with these basic 
systems, added to the various special purpose and analog 
devices, the number of unique computer configurations is 
almost infinite. To this confusing marketplace the prospec- 
tive buyer brings his enthusiasm, but not a disciplined 
approach to the selection process. Technical progress and 
new application opportunities occur so fast that any organ- 
ization's equipment strategy should come under frequent 
review. There are any number of circumstances that dictate 
the requirement for an open evaluation of computer requirements.' 

The competitive nature of the computer industry has caused 
vast technical changes during the last decade; during this 
period, the industry has moved from punched card orientation 
to online communications. Relatively fast memories and large 
capacity direct access devices with relatively fast access 
times and sophisticated operating systems, also have been 
made available with present-generation data processing sys- 
tems. Large funds, placed in research and development, have 
led to ever increasing numbers of new systems, each one 
larger in capacity, faster, more capable, with more software 
than the last ones. The industry is dynamic. The pressure 
on the present user to move from his presently obsolete sys- 
tem to the newer, more powerful system is logical and 
demanding of analysis. 

Allied to the technical changes in the data processing 
field are economic changes. Equipment is now being made 
available at substantial cost reductions. One can procure 
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a third-generation system a t 40 to 60 per cent of the cost 
of equivalent older equipment. The user who formerly made 
changes only when present equipment was incapable of satis- 
fying processing needs is now forced to compare the new 
added equipments on the basis of technical and economic 
advantages. The continual changes in the field require such 
an analysis on a periodic basis [Tatham 1969 * Yearsley 1973* 
Thrussell 1976, Joslin 1977} . 

The purchase of a computer system is a major considera- 
tion for any company, but it is often approached and inves- 
tigated in a superficial way. Computer salesmen are called 
in, but in many cases the company does not get maximum usage 
from a computer purchased from a high pressure salesman. 
Disillusionment spreads among the users, and it is quite 
often based on preconceived notions of the purchased computer. 
It is considered a "save all and do all" type deal. You can 
save all this money because the computer will do all the 
tedious work. They fail to realize that even though a 
computer can produce some material 24 hours a day, it does 
not mean that the computer is alive. Rather, the computer 
is dead simply because management is dead to possibilities. 

The manager should be completely aware of the computer's 
potential, since he finds himself sitting in on or leading 
many management teams. He should be aware of the intangible 
problems that may cause the computer to become economically 
unfeasible, or which may creep in later if the computer is 
implemented . 

Hi 



r 

v The effectiveness and potential of any computer-based 
system is strongly influenced by the design of that system 
as well as the choice of equipment. Thus the scope of the 
equipment available must be evaluated when selecting the 
equipment in order to understand how a particular choice 
affects the planned system. 

The organization entering the computer world for the 
first time normally lacks in-house talent since it may not 
have any people having background in computer applications. 
This leads to major and often complete reliance on the 
marketing wiles and brochuremanship of the computer manu- 
facturer. Like the uninformed buyer in most fields, the 
uninformed Automatic Data Processing (ADP) user normally 
contracts with the reputable firm, hoping that the firm is 
so interested in the user's unique applications that he will 
get unusual service. Thus, a major decision, capable of 
affecting the future competitive position of the firm, is 
quite easily turned over to the equipment manufacturer. A 
much better solution can be to seek outside consulting help. 
Even a substantial investment in consulting fees can pale in 
relation to the cost of a poor selection. 

The selection of a computer and manufacturer is one of 
the major decisions to be made in formulating an organiza- 
tion's computer policy. Also for the following reasons the 
decision is important; the equipment itself is probably the 
largest individual expense in the computer department, and 
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the selection of manufacturer and a specific computer sets 
a constraint on system development that will last at least 
five years [Clifton 19^9 j Tatham 1969* Wooldridge 1973, 
Procop 1976, Withington 197^ Sabol 1972] . 



11 



II. PRELIMINARY STEPS TO SELECTION 



Once it is clear that evaluation of computer needs is 
necessary under a particular set of circumstances, a series 
of studies as preliminary steps to the actual process of 
selection should be initiated. These steps are time- 
consuming, but they are an essential tool that managers can 
use to make an accurate evaluation of the equipment require- 
ments. This study is designed to determine the cost effec- 
tiveness of various solutions to the data processing problem. 
These alternative solutions will include the present system, 
plus various combinations of data processing systems. For 
each alternative, there will be a distinct systems approach, 
as well as an appropriate cost. 

Without being too ambitious and without being too 
restrictive, the planning for a data processing system must 
look at today but it must also assess tomorrow. Some of 
these tomorrow's requirements can be reasonably forecast 
today. Many of them result from growth patterns of the 
enterprise. Such growth patterns and changes are fundamental 
in nature and demand a coordinated management information 
system which can: screen all company data and prepare 

reports according to requirements, provide faster factual 
information in a dependable and management oriented manner, 
help management with both present and future information 
needs and provide data on events before they happen. 
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It is important to distinguish between what management 
wants and what it needs. The comparative analysis will allow 
management to exercise its judgment as to the feasibility of 
new system development. It is important to remember that 
this new system cannot, and should not, be undertaken on the 
basis of cost savings alone. The value of information, 
though difficult to quantify, must be considered in the 
study. The incremental cost between two alternatives will 
often be more than offset by the value that the increased 
information will give to the decision-maker. It is also 
important that these factors be weighed at high levels of 
the organization's management. 

Once an approach has been accepted by management, a 
macrosystem design effort begins. Information flows are 
defined, inputs and outputs are determined, and files are 
developed. Although all the basic data required by the 
system are defined and gross flow charts developed, the 
system designer is limited, at this point, by his lack of 
knowledge of the specific hardware configuration. However, 
it is during the applications study that the model 'which 
will ultimately become the operational system is created. 
Insufficient effort at this point can only lead to poor 
equipment selection and development of a stunted system. 

Now, failure to undergo the detailed information study can 
only mean postponement to the time when maximum effort must 
be allotted to software, procedures, and training development. 
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The importance of the application study cannot be over- 
emphasized. The results of the study are normally appended 
to the system specifications for the hardware manufacturers 
to use as a basis for proposals. Because of the importance 
in defining the model system for vendor proposals, it is at 
the applications study level that professional data process- 
ing support must be made available. Decisions on offline 
versus online systems, disc versus tape, two-channel versus 
eight-channel, basic processor speed, and memory size, 
require highly skilled systems analysis personnel thoroughly 
familiar with both the state cf the art and the function to 
be automated. Unfortunately, such personnel, almost without 
exception, are in very short supply [ciifton 1969; Joslin 
1977; Tatham 1969; Chora fas 1967 ]. 

In addition, the complexity of modern systems is so 
great that it is almost impossible for a systems analysis 
team to consider all the major alternatives. Six years 
computer experience with a company may be an excellent base 
for systems analysts or data processing management but again 
unfortunately it seldom covers experience in the process of 
computer selection or in the scale and technology about to 
be investigated. 

" The best solution to the above problem is the use of 
simulation techniques to assist in the system design. With 
this technique, a description of the user's workload (files 
and programs), along with the appropriate hardware and 
software characteristics of the configuration to be simulated, 
is used as input to a sophisticated computer program. The 
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program Is a generalized mathematical model of computer 
processing, which enables the workload to be simulated and 
valuable performance data to be collected. In this way, the 
system design analyst can try out different ideas and 
approaches in order to arrive at a good system design. v ; x 
Use of the simulation approach can drastically reduce 
the amount of elapsed time required for the applications 
study effort, and results in a better system design by pro- 
viding more analysis than is possible with a manual method. 
If this approach is taken, caution should be exercised to 
ensure that the simulation model accurately handles the 
essentials of third-generation processing. In order to be 
most useful for system design, the simulator should allow 
easy man/machine interaction by: fast turnaround time, out- 

put results oriented toward suggesting system improvements, 
and flexibility allowing design or configuration changes to 
be made easily jjoslin 1977; Graham 1973; Clifton 1969 
Martin 1973] . 
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III. SPECIFICATION DEVELOPMENT 



The topic of specifications plays an important role in 
system evaluation. Development of specifications, which will 
be released to all interested manufacturers, is a crucial 
part of the process. Specifications for a computer system 
can be prepared in a number of different ways.' The specifi- 
cations should be general enough to assure wide competition, 
yet specific enough to delineate the user’s requirements 
clearly.' The final result of the application study is a 
documented model system. This system should become part of, 
and treated by, the specifications as a point of departure 
from which each manufacturer is free to use his own ingenuity 
and brain power to develop a superior system, oriented to 
his own equipment. Proper control of this process is main- 
tained by establishing the constraints within which each 
manufacturer must work. 

The design of the system specifications should be the 
starting place for developing any computer selection plan. 

It should define what is sought in the way of a computer 
system, give the system requirements for the various appli- 
cations, and give a detailed description of each step of 
each application. The system specifications reflect the 
findings of the system study team. The final choice can be 
detrimental to both the company and the vendors if proper 
care is not taken in the preparation of the specifications. 
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The specifications must reflect the actual applications to 
be handled by the system, and must not contain poorly 
thought out limitations which are to be imposed on the 
system. Also they should not be directed too much toward a 
specific systems approach J”joslin 1911 , Chorafas 1967* 
Wooldridge 1973]. The several methods are described in the 
following sections. 

A. GENERAL SPECIFICATIONS 

General specifications are nothing more than the findings 
of the analysis, a description of the jobs to be done: the 

inputs, the desired outputs, and any other pertinent param- 
eters. (As an example of outputs, consider a stock level 
reporting. Approximately 7000 items are sold daily; for 
every item sold, a description card is produced, an excep- 
tion listing of all items below a specific amount is to be 
created daily, and a total listing of all types and amounts 
of inventory items is to be prepared monthly.) 

The general specifications give each vendor a chance to 
build a system which makes optimum use of the features of 
his system. Each vendor is free to use to the utmost any 
experience and ability he has to prepare the proposal. 

General specifications thus make maximum use of the vendor's 
system analysts and permit him complete freedom to produce 
the best possible system for the user. Rather than relying 
on the limited experience of the company's two or three 
analysts, the problems are tackled by the vendor's top 
analysts, who are more adequately geared for such work. 
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Innovations to the system may well be the result of such an 
exercise. Or, a smaller, less expensive system than was 
thought possible might be proposed as a result of some 
exceptionally good system work by a vendor. The vendor may 
propose that some of his package application programs could 
satisfy many of the system requirements. With software 
costs equaling hardware costs on today's systems, the result- 
ant cost savings could be important to both vendor and user. 

From a systems viewpoint, the user has much to gain by 
relying on general specifications to describe the applica- 
tions and needs. However, at the same time many problems 
are created. With a set of general specifications, a user 
should expect to spend many hours with vendors and their 
representatives, discussing possible systems approaches. 
Countless hours will also be spent trying to verify the 
systems proposed by the vendors in order to ensure that they 
will be able to handle the required applications. It is 
also important that the system concepts proposed are thor- 
oughly understood by the user. Whichever system is selected, 
the vendor's representative will not be delivered with the 
system. It will be the user's responsibility to turn the 
concept into reality. 

General specifications also prove awkward and difficult 
as standards by which to compare competitive proposals and 
to select a winner. With general specifications, system 
rewards can be great, in terms of improvements to the computer 
system, but the difficulties of evaluation can also be 
great [Yearsley 1973^ Chorafas 1967; Tatham 1969* Joslin 1977] . 
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B. DETAILED SPECIFICATIONS 

Detailed or specific, specifications are just what their 
name implies; each and every step to be taken in each of the 
applications is spelled out. Usually, the synthesis that 
was used in developing the flow charts for cost determina- 
tion during the system study is repeated step by step in the 
specifications. Detailed specifications must be written 
very carefully to ensure that they do net become machine- 
oriented rather than application-oriented. Machine -oriented 
specifications might discriminate against some vendors and 
thus unintentionally deprive the company of the best system. 
Since detailed specifications require the vendors to con- 
figure their systems exactly as specified, the systems design 
work for the vendors is simplified, but allows them little 
freedom to fit the applications to their computers. The 
computers must be fitted to the applications. 

Since detailed specifications are completely descriptive 
and are fully and uniformly defined to all vendors, they 
have a definite advantage for the user. Thus, there should 
be little time was ted in talking with vendors. No system 
will be proposed that is far inferior to the system repre- 
sented by the specifications. The submitted proposals may 
be more easily compared, verified, and evaluated since the 
proposed systems must all be identical, matching the steps 
set forth in the specifications. With detailed specifications. 
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the systems proposed can be no better than the system 
specified, although the trouble involved in obtaining it 
should be minimized. 

C. OTHER APPROACHES 

Specifications do not have to be either general or 
detailed; they may be at any level in the general-detailed 
range, in which a certain amount of synthesis may be done 
by the user (and the user specifies this part in detail), 
and a certain amount may be left to the imagination of the 
vendor. Such a method may be called a modified-de ta il$d 
specification. Its relative merits depend upon a rule of 
direct proportions: the more general the specifications, 

the greater the chances of obtaining a superior system; the 
greater the degree of detail in the specification, the easier 
the proposals will be to handle. The user can set the level 
of modification of a full-detail specification by the degree 
of synthesis he gives to the vendor. 

The combination of general and detailed specifications 
ought to be used in preparing system specifications. In 
this method, the general specifications are given as the 
guidelines to be followed in preparing the proposal, and the 
detailed specifications are given as an example of how the 
applications might be handled. The use of the detailed 
specification as an example serves a threefold purpose: 

1. It clearly Indicates the activities and functions to 
be performed in each of the applications, and answers many 
questions that the vendor might otherwise have to ask. 
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2. It becomes a common starting point for all vendors. 

The example may be modified differently by the contending 
vendors but they are all departing from the same basis. It 
also gives an indication of the level of sophistication 
being sought in the proposals. 

3. It gives the small vendor something good on which to 
base a bid without necessitating a full system analysis 
effort. 

Preparation of detailed specifications forces the user 
into the kind of thinking that the vendors will have to 
engage in. Since the user will be doing the thinking first, 
he should discover any problem areas before the specifica- 
tions are released to vendors. This naturally tends to make 
for smoother relations between the vendors and the user. 
Proposals submitted in response to this combination specifi- 
cation should all present solutions as good as the detailed 
specification approach [Tatham 1969, Yearsley 1973, Joslin 
1977, Chorafas 1967]. 

Usually, the type of application to be handled by the 
computer system indicates the type of specifications to be 
used. 
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IV. LIMITATIONS IMPOSED ON SELECTION 



The limitations to be imposed on the computer system 
selected should have been uncovered in the system study. 

There are two kinds of limitations: mandatory and desirable. 

The distinction between these two categories should be that 
the items listed as mandatory requirements are those items 
that are essential to the implementation of the company's 
needs . 

A. MANDATORY REQUIREMENTS 

Mandatory requirements should be stipulated by the speci- 
fications, in order to protect the user from considerations 
of proposals which will not satisfy basic needs. By defini- 
tion, a proposal will receive no consideration if it fails to 
meet any one of the mandatory requirements. The less strin- 
gent the mandatory requirements, the more likely it is that 
any given manufacturer will be able to compete in the 
procurement. Some examples of such mandatory requirements 
are shown in Figure 1. 

B. DESIRABLE VERSUS MANDATORY REQUIREMENTS 

The desirable features are only those items which would 
make the completion of the company's mission easier. Upon 
submission of a proposal by a vendor, and in consideration 
of the limitations of both categories designated by the user 
in his solicitation of proposals, the failure of the vendor's 
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FIGURE 1 



EXAMPLES OF MANDATORY 
REQUIREMENTS 



Minimum turnaround time 

Total daily processing time 

Compiler availability and requirements 

Operating system requirements 

Compa tibility 

Expansion requirements 

Translation requirements 

Site constraints 

Hardware constraints 

Equipment qualification 

Special software 
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proposal to possess some desirable feature as designed should 
invoke some penalty upon the proposal, although it would 
continue to be considered in the process of selecting the 
most advantageous proposal. 

Computer acquisitions have one thing in common: in 

most cases, the company wants the best system it can find 
for the lowest possible cost. In order to stay with the 
lowest possible cost, most users are interested in general- 
purpose computers. Special-purpose computers can be expected 
to run anywhere from 50$ to 700$ or more in cost than 
general-purpose ones [Rubin 197lJ . The very nature of a 
general-purpose computer implies certain features that are 
not readily changeable. The general-purpose computer must 
be taken largely as it comes from the plant. Thus the user 
is put into a position analogous to one who. has spent years 
drawing up blueprints for a new house, but who suddenly 
finds himself with an immediate need for a house. He can 
have the house built to his blueprints, which will prove 
very expensive due to unique building cost and the cost of 
temporary housing, or he can look for an existing house that 
fills his needs. If in searching for an already built house 
he is looking for one that exactly matches his blueprints, 
he may well have to go without a house. But if he is willing 
to not adhere strictly to the specifications, looks for 
houses with similar room layouts and other features, and 
settles upon the one most closely matching his blueprint, 
then he will have a house, which is his major requirement 
Qhorafas 1967 , Joslin 1977 > Tatham I960. 



C. LIMITING CONDITIONS 



Now that the dangers inherent in mandatory requirements 
have been discussed, the several classes of limiting condi- 
tions C3n be described. 



system is the amount of money that the company is willing to 
spend. Mandatory cost limitations may produce an effect 
opposite to the one desired. Costs should be reserved for 
use as evaluating factors and not as limiting factors. If 
a truly mandatory condition exists, such as that a fund of 
so many dollars, with provision against its increase, has 
been set aside for procurement of equipment, then the condi- 
tion should be stated. Absolute funding of this sort is 
unlikely to exist. System costs normally should be handled 
as a desirable limitation. 

2 . Due Dates 

One of the first things encountered in a system 
study will be the existence of due dates. As a matter of 
fact, these dates are not often really mandatory and they 
should not be treated as mandatory requirements but as 
desirable ones to express wishes and not commands. 



of handling the applications specified. However, the possi- 
bility remains that some of the desired capabilities are not 
available. It is better to handle application requirements 
as desirable limitations. 




A primary consideration in the study of any computer 




Application Capabilities 

The purpose of the study is to get a system capable 
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4. Responsiveness 



This is a limitation factor akin to application 
requirements. Responsiveness requirements should be regarded 
as desirable limitations, as are most application capabil- 
ities. Since responsiveness is the element of greatest 
interest to the top executives of the company, it can easily 
be made absolute by the statements of the top executives. 

5. Compa tibility 

The compatibility of the old system with the new 
should definitely be considered, but whether it should be 
mandatory is questionable. There is always temptation to 
make compatibility with the present system a mandatory 
requirement of the new system. 

6. Vendor Support 

Often seen in specifications, and stated as mandatory 
requirements, are conditions which relate to the type of 
support that must be available from the vendor whose proposal 
is accepted. They can be mandatory or desirable limitations 
depending on the particular cases of applications. 

lj Reliability 

Reliability is a condition inserted into most speci- 
fications as a mandatory limitation, generally expressed in 
such terms as "the system must have 95$ uptime". Specifica- 
tion should state that reliability will be a factor in 
evaluation and selection of a proposal, however, the value 
of reliability should not be exaggerated or expressed in 
such absolute terms as to prevent the exercise of judgment. 
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If reliability is to be used as a mandatory requirement, then 
some means of measuring it will have to be determined. Usu- 
ally the problem is that the measurement of the reliability 
would take more time than could be allowed. Or, as in the 
case of real time systems where reliability really is a 
mandatory limitation, complete redundancy of the system may 
be required. 

8 . Space Requirements 

The dimensions of systems may change from selection 
to selection. Space requirements is a desirable limitation. 
In most cases the immediately available space could be 
extended or new space found. A proposal should not have to 
be discarded just because it requires more than the allotted 
space, especially when the allotment might have been deter- 
mined arbitrarily or thoughtlessly. 

9. Input/Output Requirements 

Input/output requirements are expressed in terms of 
forms or formats to be used, number of copies to be prepared, 
and the like. Input/output requirements which are limiting 
conditions should be restudied and reappraised. They should 
be viewed in two ways: 

a. What will happen to the system if these require- 
ments are changed? 

b. If these requirements remain unchanged, can 
vendors meet them? 
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10. Other Limitations 



Many other limitations may appear in a specifica- 
tion. The deciding factor in establishing them as mandatory 
requirements is whether each item is important enough to 
warrant throwing cut the proposal completely if it fails to 
meet the limitation to any degree. This happens rarely, but 
it can happen ^Chorafas 1967* Tatham 1969 * Joslin 197 7 ’^ 
Wooldrige 1973] . 
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V. PROPOSAL EVALUATION TECHNIQUES 



A computer evaluation methodology is simply a planned 
method for selecting the most satisfactory computer system 
from a number of satisfactory computer systems. The evalua- 
tion methodology tries to assure that all of the computer 
systems in the final phase of selection are satisfactory; 
i.e., that they meet all the basic requirements of the 
solicitation and that they are what the vendors represent 
them to be. When thought of in this sense, evaluation may 
sound like a rather trivial task, since any resultant 
selection will, by definition, be satisfactory. However, 
there are different degrees of satisfaction and different 
groups of people to be satisfied. 

In evaluation, higher levels of satisfaction and satis- 
faction of other than the basic requirements are considered. 
An important group that must be satisfied with the evaluation 
process is the vendors. The vendors will not spend the time 
and money necessary to bid the ’’satisfactory systems" that 
get evaluated, unless an evaluation methodology has the 
appearance of being fair and unbiased. If they have bid but 
feel they have not been fairly evaluated, they will protest 
the selection. 

Within the government, or any large organization, a 
protest can lead to considerable embarrassment for the 
procuring activity if upheld and, in any event, will consume 
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a great deal of time and effort in resolving the protest. 
Therefore, a selection methodology must be satisfactory to 



The organization of the evaluation must be carefully 
structured so that the participants are aware of their 
individual areas of responsibility. A hierarchical arrange- 
ment is necessary in order to have increasing levels of 
responsibility as the decision areas become broader. 

In order to select the best computer system after speci- 
fications have been determined, the following need to be 
considered : 

* possible computer systems; 

* by whom the selection is to be made; 

* selection methods; 

* the criteria used in the selection process and their 
relative importance. 

The information to be used in selection methods can be 
obtained from: 

* published surveys and reports, 

* service and product publicity material, 

* hardware, operating system and program documentation, 

* managerial, sales and technical staff, 

* in-house staff and other users of the machine or service 
[Webster and Johnson 1977] . 

To give an idea about competitive selection time, 
succeeding steps in the competitive selection process for 



the vendors as well as to the procuring 




1975 
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the Navy are outlined in Table 1, which shows the estimated 
time to accomplish each step. The complexity of the system 
being acquired generally determines the length of time 
required at each step [?rokop 1976]. 

An evaluation methodology should: 

* consider those items or features wanted but not manda- 
tory, 

* cover all the items or features desired, 

* facilitate the establishment of meaningful and under- 
standable relative values between all the desired items, 

* require the completion of the previous criteria before 
the solicitation document is completed, 

* permit disclosure of all desired items and their relative 
values to the vendors, 

* incorporate systems life costing. 

An evaluation methodology that satisfies: 

* all the listed criteria is a SUPERIOR methodology, 

* five of the listed criteria should be considered a GOOD 
methodology; but before settling for it, a superior methodo- 
logy should be sought, 

* only three or four of the listed criteria may be consid- 
ered a FAIR methodology, but it should be possible to find a 
better methodology, 

* less than three of the listed criteria would have to be 
considered a POOR evaluation methodology and should not be 
used [Auerbach 1975, House 1976, Chorafas 1967] . 



31 



TABLE 1 



TYPICAL COMPETITIVE SELECTION TIME FRAME 



Draft request for proposals for 
approved project 

Release of draft for comments 

Revision of request for proposals 

Response to request for proposals 

Evaluation of proposals and benchmark 

Administrative time after evaluation 

Installation of equipment after 
contract award 

Range: 290-720 Days or 10-24 Months 



30-90 Days 

30 Days 
30 Days 
60-120 Days 
30-120 Days 
20-60 Days 
90-270 Days 



Prokop 1976 
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There are several techniques for the evaluation of compu- 
ter systems. They generally fall into one of two categories. 



Either they are very simplistic in that they tend to ignore 
most of the criteria listed previously, or they are sophisti- 



1 . Simple Techniques 

Simplistic methodologies are better known but less 
successful techniques. Theoretically, they are not worth 
much discussion, but they illustrate the need for the more 
sophisticated techniques. 

a . Sole Source 

"I have been happy with this vendor. Why should 
I change?" It is possible that one might be happier, for 
less money, with another vendor. 

b. Subjective Judgment (Overall Impression) 

Probably the most frequently used evaluation 

approach is to have no preestablished approach, just some 
general statement such as: "When the proposals come in, an 

unbiased group of evaluators will look through them and pick 
the one that provides the most benefits at the lowest prices." 
People who advocate this approach to computer selection will 
ridicule any attempt by a prospective user to preestablish 
which items he must consider in an evaluation before he 
receives the proposals to be evaluated. They will ridicule 
the prospective user even more if he attempts to establish 
specific values for each of these items. 



cated and incorporate most of those criteria 




Prospective evaluators will argue that until the 
prospective user has received all the proposals for computer 
systems, he will not know all the items. They will point 
out, for example, that if all the vendors propose any given 
major item, then its importance to the selection process is 
negligible; whereas even a minor item can have significant 
influence on the final decision if it is proposed by one or 
two of the vendors but not by all. These prospective evalu- 
ators want to make their selection first and then justify 
their evaluation. 

This procedure is a comfortable one for the 
evaluator since he is not forced into doing any advanced 
planning. Almost any vendor who meets the mandatory require- 
ments could be selected as the winner under those circum- 
stances. All it takes is an evaluator who is clever with 
words and who can accentuate the strong points of the winner 
and flaunt the weaknesses of the losers. However, it is 
unfair to the vendors 'and, in the long run, also unfair to 
the prospective user to establish the criteria for evaluation- 
after the proposals are received. This kind of evaluation 
is subject to the vagaries of human nature, over which there 
is no control (joslin 1977 j> Rubin 197l] . 

c. Cost Only 

f ~ 

This technique advocates selecting the lowest 
cost system that meets all of the mandatory requirements. 
However, what if the next-to-the-cheapes t system is only 
slightly more expensive than the cheapest one, and yet would 
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far outperform it? Due to the unanswered cost and require- 
ment questions, the cost-only approach is rapidly losing 
favor, except for smaller systems with static workloads 




Any meaningful evaluation methodology should 
differentiate between mandatory and desirable features. 

Either a vendor shows that he can meet all the mandatory 
requirements or his proposal is not considered for evalua- 
tion. If only one vendor can satisfy all of the mandatory 
requirements, he is automatically the selected vendor. 

Suppose, however, that three vendors were to 
satisfy the mandatory requirements, then the proposals of all 
three would be considered to be equally satisfactory. The 
purpose of the evaluation methodology is to establish some 
logical and defensible means of differentiating between the 
proposals of these satisfactory vendors and selecting the 
one that is best suited to meet the activity's needs. The 
items used to differentiate between the vendors who have 
satisfied the mandatory requirements are desirable require- 
ments. Since there are many desirable features of varying 
importance, the evaluation methodology must find some method 
of establishing the relative values of these features and 
their relationship to the system cost, 
d. A Case History 

Company ABC wanted a computer system. The 
company needed to have a given set of problems processed 
within one hour's time, and ‘there were certain other 
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requirements that had to be met. ABC sent out a request for 
proposal, to which five vendors responded. Three proposals 
satisfied all requirements. The only significant difference 
between the three proposals was the amount of time it would 
take to process the problems and cost of the three different 
systems. The findings -were : 



Vendor 


System Cost ($) 


Time to Complete Problems (Min) 


X 


300,000 


50 


Y 


275,000 


55 


Z 


500,000 


25 



Vendor Z was selected by the evaluators because 
they interpreted the time to process the problems in its 
reciprocal sense of how many sets of problems could be proc- 
essed per hour. Thus, vendor X could process 1.2 sets; 
vendor Y could process 1.1 sets; and vendor Z could process 
2.4 sets. They then divided each system's cost by the number 
of sets which the system could process, with the following 
results : 



Vendor Cost Per Set Per Hour 

X $250, 000 

Y 5260,000 

z $208,333 

The evaluators justified their choice by pointing 
out that vendor Z's system gave the most room for expansion 
and could process more sets of problems per dollar than either 
of the other systems [Auerbach 1975] . 
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2. Sophisticated Techniques 

There are two basic evaluation methodologies which 



permit the evaluator to consider desirable features and 
establish the relative value of the desirable items. The 
approaches are weighted-scoring schemes and cost-value based 
approaches . 



a. Weighted-Scoring Technique 

Under this system, the prospective user preassigns 



varying quantities of points to all items he considers impor- 
tant and then selects the system earning the most points. 

An example of this technique is shown in Table 2. Vendor B 



the criteria listed, it may seem to be a good evaluation 
methodology. However, upon closer review it is found to fall 
down on the criterion which calls for "meaningful and under- 
standable" relative values between all desired items and on 
the other criterion which calls for incorporation of system's 
life costing. (See page 30 for the criterions.) 



understandable relationship between the number of points 
awarded for low cost. For instance, the example shows cost 
having a weight of 70$, but why 70 rather than 30 or 50 or 
90 $? The failing of this technique is that there is no 
common denominator among the items being weighted. Thus, 
there is no "meaningful and understandable" relationship. 



would be selected 




Since this technique appears to satisfy all of 



It is difficult to establish a meaningful and 
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TABLE 2 



AN EVALUATION USING WEIGHTED SCORING 





Re la tive 


Vendors 1 


Scores 


Evaluation 
I terns 


Values 

* 


A 


B 


C 


Cost 


70 


70 


60 


50 


System potential 


20 


7 


lo 


20 


Technica 1 
characteristics 


5 


2 


4 


5 


Vendor support 


3 


3 


4 


5 


TOTAL SCORE 


100^ 


82 


84 


80 
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Until a meaningful approach can be found to the proper 
distribution of points between the items desired, the 
weighted-scoring technique will never be considered a very 
satisfactory evaluation technique. 

b. Cost/Effectiveness Ratio 

This technique is simply a subcategory of the 
weighted-scoring technique, except that with it, by dividing 
the total system cost by the sum of the points scored in the 
other desirable categories (effectiveness category), the 
prospective user can select the system with the lowest ratio 
of cost to effectiveness. However, such a division of 
points is generally not sufficient to establish a meaningful 
relationship between cost and effectiveness fjoslin 1977 , 
Borovits 1975 ] . 

c. Cost-Value Technique 

None of the previous evaluation techniques 

proved very satisfactory under intensive investigation. 
Therefore, a new evaluation method, the cost-value technique, 
was developed in 1964. This technique combined the simplic- 
ity of the cost-only technique with the realism of the 
weighted-scoring technique. The result was a technique 
superior to both. It is superior to the cost-only technique 
because it considers some items in a computer system to be 
of value in addition to the system's cost and its compliance 
with the mandatory requirements; and it is superior to the 
weighted-scoring technique in that it establishes a meaningfu 
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relationship between the items of value and the system’s 
cost while at the same time incorporating system's life 
costing (chorafas 1967 .. Joslin 1977] . 

The cost-value technique recognizes the necessity 
of evaluating the desirable features offered by the various 
computer systems proposed. With this method, the desirable 
features and the cost associated with the system are all 
that are evaluated; that is, the ability of the proposed 
systems to perform the functions for which a computer is to 
be procured and the ability of the vendors to meet any other 
conditions specified as mandatory in the specification 
package are not evaluated, but are validated. If it is 
found that the vendor or his system cannot perform as 
required, the proposal is eliminated from further considera- 
tion. With this technique a company can study any extra 
features offered in the proposals to determine whether the 
claimed extra features are important in themselves or are 
mere incidental elements that appear to be extra features. 

For example, a 60-nanosecond memory and a 10, 000-card-a -minute 
card reader are not important features in themselves. More 
important and desirable is the amount of slack time that 
exists in the proposed system on account of these high-speed 
units. A study should be initiated to determine the value 
of every extra feature which is considered to be important. 

A distinguishing feature of the cost-value 
technique is the assignment of the value associated with the 
desirable features in terms of cost, that is, dollars, of 



40 



value. 3y assigning a cost value, in dollars, to the desir- 
able features offered by the various vendors, a common 
denominator is provided by which all offered desirable 
features may be related to each other and to the system’s 
cost. Although the cost values assigned to the various 
desirable features still will be a matter of each individual 
selection, and will continue to reflect the opinions of the 
assignors, a value, when assigned, can be understood, examined, 
discussed, and changed independently of all other individual 
assigned values. 

An important benefit derived from the use of 
assignment by cost value is that management can understand 
what is going into an evaluation, and is able to make 
informed decisions on the value of any disputed desirable 
features. The specific cost values established for each of 
the various desirable features found within each of the 
proposals are then used for the scoring of the proposals. 

In the cost-value technique, the proposals are scored or 
ranked by what will be referred to as a cost-value account- 
ing scheme. This is cost and value accounting, since 
some of the values and costs used, although stated in dollar 
terms, may net involve real expenditures. 

The cost-value technique amounts to taking the 
total cost of a system proposed and then deducting the cost 
values of all the desired extras included in that proposal. 

The difference represents the derived cost of satisfying the 
mandatory requirements stated in the specification package. 



41 



The system having the lowest derived cost for satisfying the 
mandatory requirements becomes the system selected, since 
the values of desirable features offered already have been 
taken into consideration in deriving this cost for satisfying 
the mandatory requirements. This ranking also can be looked 
at from a value-to-cost ratio, but the results will be the 
same if value is considered in its full sense, as value of 
mandatory requirements plus the value of the desirable 
features offered, and cost is considered to be the total 
cost of the package over the estimated life of the system 
jjoslin 1977 , Tatham 1969, Auerbach 1975]. 

(l) Using The Cost-Value Technique . The cost- 
value technique's approach to the extra features (those 
proposed features above and beyond the mandatory require- 
ments likely to be offered by the vendors) is to appraise 
them to determine whether they are worthy of inclusion in 
the evaluation, and, if so, to determine the cost value of 
these features. To avoid any bias or appearance of bias on 
the part of the evaluators, and in order to be fair to both 
vendors and the user, this study preferably should be initi- 
ated before the proposals are received. However, it should 
be noted that the cost-value technique actually is open- 
ended; that is, if any unexpected extra features are offered, 
they can be included as part of the selection, if deemed 
important enough, by simply assigning them their cost value. 
It thus becomes necessary to deal with either hypothetical 
or realistically anticipated features. A sample listing of 
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features which may be considered in cases must first be 
established; then it can be seen how to go about assigning 
cost values to those items which really have value to a 
selection. Table 3 contains a sample listing of these 
features . 

COST ITEMS. All cost items must be consid- 
ered in the evaluation. Items such as the cost of supplies 
or personnel may prove to be nondifferentiating in a given 
selection, but they should not be deleted from the evalua- 
tion list because, differentiating or nondifferentiating, 
they are still true costs associated with completing the 
specific applications. 

Treating cost items as one-time costs or 
continuing costs is a matter of cataloging. The following 
rules must govern any proper treatment of cost items [joslin 
1977]: 

1. The costs must be spread proportionately over the 
expected life of the system. 

2. The system costs must change to reflect the costs of 
any planned system expansion. 

For example, if the life of a system is set 
at six years and a uniform expansion rate of ten percentage 
per year is expected over the life of the system, then each 
of the continuing cost items on the list, if applicable to 
the yearly system cost, should be charged for six years. 

Thus it would be expected that the equipment cost for the 
sixth year would be larger than that for the second year. 
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TABLE 3 -a 




44 



TABLE 3-b 



COSTS 



ONE-TIME COSTS 

Site Preparation 
Elec trica 1 

Air conditioning (cooling, heating, and humidity 
control ) 

Power supply (including all wiring) 

Space for equipment 

Facilities (walls, ceiling, painting, draperies) 

False flooring (including bracings) 

Security provisions 

Equipment Installation 

Equipment Transportation (including insurance cost) 

Vendor Support 

Personnel (analysts, programmers, operators, instructors) 
Training (including transportation, living costs) 

Existing programs 
Backup facilities 
Machine time (checkout) 

Documents tion 

Program and data conversion 
CONTINUING COSTS 

Procurement of Computer System Equipment (falls in one-time 
costs category if system is purchased) 

Central processor and associated equipment (console, 
floating point option, real time option, etc.) 
Peripheral computer equipment: on-line or off -line 

(remote-inquiry device, card reader, printer, etc.) 
Auxiliary equipment 

Keypunch machines and other data -created devices 
( flexiwriter, teletype machine, etc.) 

Printers, sorters, collators, etc. 

Operation and Maintenance of All Electrical Equipment 

Personnel (manager, analysts, programmers, operators, etc.) 

Program Development 

Supplies (magnetic tape, printer paper, cards, etc.) 

Indirect Cost for Space Used 
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TABLE 3-c 



EQUIPMENT CHARACTERISTICS 



SPEED 



Time required to complete applications specified 
Instructions 

Add time (fixed and floating) 

Mult, time (fixed and floating) 

Divide time (fixed and floating) 

Move 

Other instructions (through all other instructions 
thought significant) 

Peripheral equipment 

Printer (lines per minute) 

Card reader (cards per minute) 

Card punch (cards per minute) 

Magnetic tape units (characters per second) 

IAS (characters per second, average) 

Other equipment (through all other peripheral equipment 
listed) 



CAPACITY 

Storage capacity of main memory (core) 

Storage capacity of immedia te -a ccess storage (IAS) 
Storage capacity of magnetic tape 
Characters per printed line 

COMPATIBILITY 

Program; tapes; cards 

RELIABILITY 



Error detection; error correction techniques; mean time to 
failure, etc.; redundant components 

SPECIAL FEATURES 

Memory lockout; parallel processing 
PROBLEM TIMINGS 

Central processor limited; input/output limited 
SWITCHABILITY 

Magnetic tape units; printers 
OTHER CHARACTERISTICS 

Size of equipment (each piece considered); weight of 
equipment (each piece considered) 
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TABLE 3-d 



EXPANSION POTENTIAL 

SLACK TIME (amount of available free time on each piece of 
equipment) 

Central processor., magnetic tapes, immediate access storage, 
card punch, printer, remote terminals, etc. (through all 
other equipment offered) 

MAXIMUM EXPANSION (number of units that can be added to 
system ) 

Magnetic tapes, immediate access storage, card punch, printer, 
etc. (through all other system equipment offered), extra core, 
disk drives 

COMPATIBLE EQUIPMENT 
Larger processors 
Higher performance units 
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TABLE 3-e 



VENDOR'S SUPPORT OF SYSTEM 
PROGRAM ASSISTANCE 

Development; writing; converting; emulating 
TRAINING 

Analysts; programmers; operators; managers; users 
MAINTENANCE OFFERED 
BACKUP AVAILABILITY 
PROGRAM TESTING 

Hours; shift; location 
EXISTING SOFTWARE 

Operating system 

Schedulers; input/output control; memory allocation 
etc . 

Sort; merge; system simulators or emulators; COBOL; 
FORTRAN; report generator; etc. 

DOCUMENTATION 

PERSONNEL LOANED 

Analysts; programmers; operators; users 
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Another important consideration relating to 
cost items is that they should show the cost for individual 
pieces of equipment to be used. This should be true in all 
cases except when the system is to be used for less than one 
shift, or when the entire system is to be purchased. 

No cost items should be duplicative; that 
is, the system should not be charged twice for the same equip- 
ment or service. For example, if a card reader is used both 
online and offline, its full cost should not be shown twice. 
Similarly, program development, if performed by the user's 
personnel rather than the contractor's personnel, and person- 
nel cost should not both be costed for this program. 

EQUIPMENT CHARACTERISTICS. The significance 
of the characteristics of any piece of equipment is measured 
in terms of the running time of the system, which in turn 
determines the system's cost and expansion potential or its 
system responsiveness. 'Typical of the kind of equipment 
characteristics now being discussed are: the relative speeds 

and capacities, hardware compatibility, switchability, relia- 
bility, and special features. For real time systems, these 
conditions usually will be stated as mandatory requirements 
[Chora fas 1967* Coutinho 1977J . 

Sample problem timing items (times required 
to perform the benchmark problems) should not be evaluated. 
They should be used exclusively for validation or establish- 
ment of application timings quoted in the proposals. This 
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application timing, in turn, is the base on which system 
cost and expansion potential are calculated. Since its 
value is felt in the other items, it should not be evaluated. 

EXPANSION POTENTIAL. The expansion poten- 
tial of a system is considered to be an important extra, 
since it allows for growth beyond the specified amount. Thus 
the system has the possibility of a longer life and of han- 
dling larger workload peaks. Another type of expansion which 
is sometimes important is the ability to add on different 
types of peripherals. 

VENDOR SUPPORT. Vendor's support is also 
deemed important to the cost-value technique. All of the 
items of vendor's support could be desirable features, since 
each offer could result in some actual saving to the user 
[Thrussell 1976, Sabol 1972], 

(2) Constructing Evaluation Templates . The 
cost-value technique examines expansion potential by 
evalua ting : 

1. The system's ability to handle additional workloads, 
and 

2. The system's ability to handle different peripherals. 

To evaluate the system's capability for 
handling additional workloads, it is necessary to first 
calculate the run time required by the system to complete 
all required applications. The elements and aspects of a 
computer and its use that must be considered in any calcula- 
tion of the running time of a system are: 
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1 . 



Speed 

a) Central processor 

b) Peripheral equipment 

c) Auxiliary equipment 

2. Capacity 

a) Central processor 

b) Peripheral equipment 

3. Special features 

a) Parallel processing 

b) Simultaneous operations 

c) Other 

4. Software efficiency 

a) Compiled languages 

b) Assembled languages 

5. Reliability 

a) Switchability 

b) Error detection and correction features 

6. Preparation time 

a) Setup/take-down 

b) Program insertion 

c) Media handling 

7. Nonproductive time 

a ) Reruns 

b) Program checkout 

If a system were used 24 hours a day for 30 
days each month, it would be possible to get 720 hours a 
month of computer time. However, most manufacturers require 
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about hours a day for preventive maintenance of the 
machine and this would leave about 675 hours a month. From 
this 675 hours must be subtracted various loss factors, such 
as unscheduled maintenance, idle time, setup time, machine 
malfunction time loss, program housekeeping, development and 
maintenance of programs, program errors, and operator errors. 
These reductions cut the actual maximum time available for 
production to between 400 and 500 hours per month for most 
business applications and 500 to 600 hours per month for 
scientific applications [joslin 1977* Webster and Johnson 
1976]. 

The figures for total production time 
available should be used to deduct the times computed to 
process the monthly workload. The time remaining, called 
slack time, is the time available for expansion. The amount 
of slack time available could be increased by reducing the 
time required to process the monthly workload, which could 
be achieved by adding processors of higher capability. 
However, more or faster units should be added only when the 
value of the additional slack time is greater than the cost 
of modifying the system to make this additional time avail- 
able. The worth of the additional slack time might be 
considered as the additional system life brought about by the 
expansion potential. The concept of system life applies not 
only to a purchased system, but also to a leased system where 
there is extensive investment in software and know-how which 
is geared to the existing system. Any changeover thus might 
prove costly. 
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YEARLY EXPANSION. The most meaningful way 
of preparing a value template for yearly expansion would be 
to look at the stated workload for each year, estimate the 
confidence level that the stated workload is correct, then 
increase the workload until a confidence level of about 95$ 
is obtained. For example, if it were estimated that the 
stated workload for the first year was 100$ of some base 
amount, after considering the case it might be found that 
there was only about an 80 $ confidence in that estimate. 
However, if that base amount were increased to 110$ of the 
old base (having a confidence level of 30$), there would be 
88$ confidence; if it were increased to 120$ of the old base, 
there would be 96 $ confidence. The estimate would have to 
be increased to 125$ of the old base before there would be 
100$ confidence that the workload as then stated could not 
be exceeded In the first year [joslin 1977] . 

Assume that the system envisioned for this 
case was expected to lease for $100,000 a year. With these 
facts, the following value template might be established for 
the value of expansion for the first year: 

First Year Expansion Value Template 



Confidence Level 

Percentage Expansion Value 

75 $ 20,000 

20 $16,000 

10 $ 8,000 
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The value figures are derived by saying 
that, if the user were willing to pay $100,000 to handle 
what he is only 80 $ confident represents the first year's 
workload, he should be willing to pay 25$ more to have 100$ 
confidence in the system's ability to handle all the first 
year's workload; that is a total of $120,000, or an increased 
value of $20,000. 

Similarly, to be 96 $ confident rather than 
80 $ he ought to be willing to pay 20 $ more, or $ 16 , 000 , and 
so forth. In a similar way, an evaluation template could 
be prepared for each of the years. 

If the evaluation templates are supplied to 
the vendors, there should not be any need to adjust the 
vendors' proposals to reflect the greatest value for the 
user. If the vendors are not supplied with the evaluation 
templates, then the value of expansion potential must be 
calculated for each year. For example, if a vendor were to 
propose a system that was so modulated that every year his 
system took all the time available just to handle the 
required workload, but examination of his equipment revealed 
that he could increase his system's capability by 10$ for a 
yearly lease increase of $2,000, or 20$ for an increase of 
$6,000, or 75$ for an increase of $13*000, adjustments to 
his value of expansion potential should be made. 

If the previously established Value Template 
were to be used, the vendor's yearly values for expansion 
would be : 
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Increased Value Ratio To Increased Lease Price 

$ 8, 000 $ 2 , 000 

Sl6,000 $ 6,000 

$ 20,000 $ 13,000 

The best difference of value minus cost is $16,000 - $6,000 
= $10,000; a 20$ confidence expansion is indicated. 

A slightly different approach for determining 
the relative value of yearly expansion could make use of mar- 
ginal utility analysis techniques, but similar results should 
be obtained £we bster and Johnson 1976, Thrusseil 1976, Joslin 
1977]. 

EXPANSION BY NEW OR DIFFERENT PERIPHERALS. 
There are times when it is of definite value to be able to add 
peripherals to the system that were not called for in the 
basic system requirements. For example, it might be possible 
to handle a given application without using immediate access 
storage. However, if the user feels that sometime in the 
future he might wish he had experience with immediate access 
storage (IAS), he might establish a value for having the 
system possess the capability of connecting as IAS unit. In 
fact, he might establish two values: the value of having IAS 

proposed and a lesser value of having the capability to add 
an IAS unit. 

The suggested method for determining the 
value of such a capability is: 

1. Calculate the probability of needing the capability. 

2. Determine the cost of obtaining the capability inde- 
pendent of the present system. 

3. Take the product of these two figures. 
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It must be remembered that there is likely 
to be considerable difference in the evaluation of the 
various capabilities proposed, since no one application 

measures every possible capability. 

EXPANSION WITHIN A FAMILY. The advantage 
of this type of expansion is that the programs written for 
one of the smaller computers in a family will run on the 
larger ones. Therefore, the only expansion cost is that of 
the new computer, not a reprogramming cost. To the extent 
that this statement is true, the family approach could be 
used in a fashion similar to the extra system life approach. 
However, the inefficiencies of running programs on a large 
computer that were prepared for a smaller one must then be 
considered. 

VENDOR SUPPORT. There are several methods 
of assigning cost values to vendor support items. The 
simplest method, for which the user cannot estimate the 
value of the support items, is to require the vendor to 
quote costs for various levels of performance. For example, 
if one vendor offered on-site maintenance while all the rest 
offered on-call maintenance, a feeling for the maximum value 
of the on-site maintenance could be ascertained by asking 
each of the other vendors to state the cost of such service. 
Sometimes, however, the cost quoted may be so excessive as 
not to make a fair base against which to award value. For 
example, if a user were impressed by some special program- 
ming routine and asked various vendors for the cost of 
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supplying it, he might receive answers in hundreds of thou- 
sands of dollars, whereas if he himself were to go out and 
procure such a routine, he probably would not be willing to 
pay over $5; 000. In such a case, the $5^000 should become 
the base. Restated as a generalized rule: In cases where 

the user would place a value on a service lower than a 
vendor's cost, this value figure becomes the base for deter- 
mining the item's value [joslin 1977). 

In some cases, the vendor may not be able 
to give cost figures for supplying service equal to some of 
the levels desired, simply because he does not have the 
necessary facilities. In other cases, it might be practical 
only for the vendor himself to provide the service. An 
example of this is a special training requirement which might 
occur in a real time system, where some provision, probably 
a special program, must be provided to allow the trainees 
access to the remote consoles for training purposes, yet 
prevent their mistakes from destroying the good system. This 
kind of training aid probably can be provided only by the 
given vendor. In such cases, the cost value of such a 
service must be determined individually, and might be con- 
siderably higher than the costs charged by any other vendor. 
But the higher cost-value figure should become the base. 

The cost value of these items also might be 
ascertained by the user, by taking each item in turn and 
determining its value to him. Among these cost-value items. 
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the one most closely representing the user's needs should be 
chosen. However, the cost value should never exceed the 

1 

cost of having the service contracted by someone else. 

Some items such as available backup and 
debugging facilities are support items on which the vendors 
cannot be asked to change or improve. Therefore, their cost 
value must be evaluated as the items are proposed. An 
approach to determine the cost value of back-up would be to 
determine the probability of experiencing a catastrophic 
failure, then the cost associated with carrying on the 
computer activities on the back-up facilities available. 

The cost times the probability of catastrophe should give 
the probable value for back-up of each of the various systems. 
Cost-value determination for debugging facilities could be 
handled in the same way [Chorafas 1967 * Sabol 1972]. 

OTHER DESIRABLE FEATURES. Many other 
features might be considered in hardware selection. Items 
such as memory lockout or desirable compatibility can be 
handled by determining the cost that will be eliminated by 
the inclusion of such abilities. Thus the costs that would 
have to be paid to convert tapes of one kind to another 
would be saved if the two systems were compatible. This 
cost becomes the cost value. 

Being able to run a portion of the old 
programs on the new system is a desirable feature. Therefore, 
program compatibility (or portability) is another important 
aspect of compatibility. An estimate can be made of the cost 
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that would be incurred if that portion of the software had 
to be run elsewhere until rewritten. This estimated cost 
becomes the cost value of program compatibility. However, 
if the compatibility is achieved through the use of an 
emulator or simulator and the resultant programs would not 
run at the efficiency of rewritten programs, then the value 
of this compatibility is decreased. The amount of the 
decrease would be dependent upon the frequency of use of the 
programs. Infrequently used programs do not need to be as 
efficient as frequently used programs. 



saved by inclusion of a memory lockout device become its 
cost value, which can be shown in an evaluation template 



management to have access to any information within the file 
in less than one minute, or to have management reports ready 
by 1:00 p.m. everyday. In such cases, a study must be 
initiated to determine the cost value to management of being 
able to have one-minute access, rather than ten-minute 
access as requested in specifications package, or the cost 
value of having the reports ready by 1:00 p.m., rather than 
3=00 p.m. as similarly requested. Where possible, these 
value assignments should be made in time to help the vendors 
with their bidding. 

With real time or time-shared systems, 
another area of desirable features should be considered. 



Costs of the time and trouble that could be 




A system may be proposed that will enable 
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For example, a time-shared system may call for eight remote 
terminals with an inquiry from any terminal being handled 
within six seconds. Some of the systems proposed may be able 
to handle two or three times the required number of termin- 
als. The value of these extra terminals depends upon the 
probability of their profitable use or on some logic similar 
to that used in making the original decision that eight were 
required. The value of various speed responses should be 
determined and shown in an evaluation template. 

Another extra is the possibility of coming 
across an innovation or a new approach to the system. The 
prospective user could assign a cost value to such a new 
approach by making a realistic determination of the savings 
that are likely to accrue if he uses the suggested approach 
times the degree of probability that the suggested approach 
will actually work, or by estimating how much it would have 
cost him if he had had a special study made that might have 
come up with the same recommendation. 

Extras such as a purchase option offered or 
expected trade-in will or will not have value, depending 
upon the type of procurement plan to be used in the acquisi- 
tion of the system. 

(3 ) General Thoughts On The Cost-Value Technique . 
In applying the cost-value technique, the following points 
should be kept in mind: 

1. The methods described here for cost-value determinations 
are by no means the only ones that might be used. 



60 



2. The items chosen for discussion are not the only items 
worthy of inclusion in a cost-value evaluation, nor will 
they all necessarily appear in any given selection. The 
circumstances of a specific selection determine the items to 
be used. 

3. There is nothing sacred in any of the cost values 
established, since the value of any item depends upon the 
likelihood of the user's need for that item. For instance, 
if the described system is to be used only for one or two 
applications and the size and volume of these applications 
are fixed then the cost value of expansion potential is 
likely to be nil. On the other hand, if the described 
system is the first system to be installed in a growing 
company, the cost value of expansion potential will be high 
because every hour of available expansion might be as valu- 
able as each hour in actual use. If the described system is 
to replace an existing, compatible system, some of the 
vendor support items, such as personnel loaned or program 
assistance, may have no cost value. However, if the computer 
is for a relatively inexperienced group, such factors might 
have a cost value as high as, or higher than, $40,000 per 
man-year [joslin 1977] . 

Calling the items to be evaluated "extras" 
implies that the "extra" is the amount over the minimum 
acceptable or mandatory. The value of an extra may be estab- 
lished independent of the proposals from preconceived values 
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which are created to show value for varying amounts of each 
of the extras evaluated. These preconceived ideas of worth 
are referred to as "evaluation templates". 

ESTABLISHING LIMITS. To assess the value 
of any given item is a difficult task. The logical starting 
place is what the item costs. If the item is competitively 
available, its value should never greatly exceed its cost. 

For example, if the cost of having a mathematical subroutine 
written by a softvrare consulting group is $10,000, then it 
would be reasonable for a user with little use for this 
subroutine to establish a value of only $500 for its 
availability. 

If only one vendor can supply a critical 
subroutine, its value is almost indeterminate. However, 
this case should not arise in cost-value assessment for two 
reasons : 

1. If the item is critical, it should be listed as manda- 
tory requirement and should not require value assessment. 

2. If the item can be procured from only one vendor, the 
full selection ought to have been handled as a sole-source 
procurement, again making value assessment unnecessary. 

DIMINISHING VALUES. The prevailing thought 
behind most of the evaluation templates created is that, as 
more of an item becomes available, the worth of that item 
decreases. 'This can be shown mathematically by exponential 
curves such as these shown in Figure 2. However, for ease of 
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understanding, it is usually easier to break down each value 
assignment into a group of smaller approximations (Chora fas 
1967 , Tatham 1969 . Sabol 1972, Joslin 1972J • 

Suppose that the ability to be able to 
expand by 20$ is worth $20,000; by ^0$, $ 32 , 000 ; by 60 $, 
$40,000; by 80$, $46,000; and by anything over 100$, 

$50,000. An exponential curve could be fitted through these 
points and the curve established, but it is usually not 
worth the effort. Using normal interpolation techniques 
between the defined points, the value of any expansion 
capability can be found. Thus, the value of having 50$ 
expansion capability could be found by taking: 



Difference between 

40$ and 50$ 10$ 



x 



Difference in value 
for 40$ and 50$ 



Difference between 
40$ and 60 $ 



20$ $8000 Difference in value 

for 40$ and 60 $ 



The unknown difference is found to be $4,000. When this 
amount is added to the amount for 40$, the resultant value 
for 50$ is found to be $36,000. Figure 3 shows this template 
plotted, using straight-line extrapolation between the 
defined points and using an exponential curve. The value 
for 50$ expansion, if taken from the exponential curve, would 
be approximately $37*000. 

An explanation has been given of some tech- 
niques for determining the cost values of a number of items 
that should be included in any selection. The cost values 
derived for the various vendors are applied as credits to 
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Value (A ~ ZOOOO) 



FIGURE 2 



True Exponential Siiape of Normal Evaluation Templates 

rfd A B 




Feature* 



FIGURE 3 



Evaluation Templates 




offset the costs of the system and the proposed services. 

The vendor whose proposal shows the smallest difference in 
out-of-pocket costs minus credits is the one to whom the 
contract should be awardel^ 

d. Requirements-Cos ting Technique 

This technique is conceptually the same as the 
cost-value technique, only under this approach a vendor is 
assessed a preestablished dollar value or worth for any 
desirable feature not offered (or offered at a cost that 
exceeds its worth) by the vendor in its proposal; or if the 
vendor offers the feature, but at some charge, then the 
vendor is assessed that charge. The system selected is the 
one having the lowest overall total cost (including not only 
the cost of the vendor's hardware, software, and services, 
but also other costs for such items as staffing, power, air 
conditioning, etc., and assessments for features net offered). 
An example of requirements costing is shown in Table 4. 

The requirements-cos ting technique and the cost- 
value technique are essentially identical, and they prove 
exceptionally satisfactory once the dollar values of the 
desirable features are established. Both of these techniques 
meet all the criteria listed as essential for a superior 
evaluation methodology. 

e. Dynamic Approach 

The problem and an approach to a solution is 
presented here in the form of an example. Assume an organi- 
zation which has decided to replace its existing computer 
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TABLE 4 



AN EVALUATION USING REQUIREMENTS COSTING TECHNIQUE 





Maximum 


Vendors ' Cost 


$ 


Eva lua tion 


Values 








I terns 


$ 


A 


B 


c 


Vendor Costs 




1,000,000 


1,200,000 


1,300,000 


Other Costs 
Assessments 




100,000 


95,ooo 


90,000 


System 

Potential 


400,000 


240,000 


100,000 


20,000 


Technical 

Characteristics 


200,000 


60,000 


35,000 


10,000 


Ve ndor 
Support 


50,000 


40,000 


25,000 


0 


Total Cost 




1,440,000 


1,455,000 


1,420,000 



Auerbach 1975 



66 



system in preparation for a major expansion in its informa- 
tion processing activity, proposals are on hand from three 
manufacturers. A, B, and C, each of whom has a number of 
systems to offer, named Al, A2,...C4 and benchmarks have been 
performed for a representative sample of the organization's 
workload on one or more systems of each manufacturer. 

The least powerful system, Al, has been chosen 
as a reference point, and its capacity assigned a value of 
1. Based on the benchmark runs and extrapolations from them, 
the capacities of the remaining systems have been determined 
as multiples of the capacity of Al and tabulated as in Table 
5-a, where each column represents systems of comparable 
capacity. From the table, it follows that system A2 of 
manufacturer A is 1.8 times more powerful than system Al, 
and so on. 

The rental prices of the various systems detailed 
above, including software charges, are shown in Table 5-b. 
Dividing the data of Table 5-a by those of Table 5-b provides 
an indication of the capacity per dollar outlay, which can 
be called cost-effectiveness, for each of the systems. These 
figures are shown in Table 5-c. Assume that the firm faces 
an anticipated growth in workload as shown in Figure 4. Year 
0 on the horizontal axis is the current year, and year 1 is 
the year of installation. The planning horizon considered 
is six years. The vertical axis represents anticipated 
workload expressed as multiples of the workload capacity of 
the reference system, Al. The planned major expansion in 
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TABLE 5 -a 



System 



Manufacturer 


1 


2 


3 


4 


A 


1.0 


1.8 


2.6 


3.5 


B 


1.2 


— 


— 


3.3 


C 


— 


1.4 


2.3 


■? ii 

* 



Manufacturer 


TABLE 


5-b 

System 






1 


2 


3 


4 


A 


$60 


OJ 

CO 


$100 


$115 


B 


$77 


— 


— 


$97 


C 


— 


$82 


$98 


$116 




TABLE 


5-c 










System 






Manufacturer 


1 


2 


3 


4 


A 


1.67 


2.20 


2.60 


3.04 


B 


1.56 


— 


— 


3.40 


C 


— 


1.71 


2.35 


2.93 



Ein-Dor 1977 
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information processing activity causes a rapid climb in the 
curve in years one through five with a subsequent return to 
stability in year six. 

At the end of six years, for this system, the 
workload will be about three times the capacity of the 
reference system. Under the conventional form of computer 
selection, a system should be chosen which either satisfies 
immediate requirements and is evaluated as having good 
growth potential or satisfies requirements for the next five 
or six years. The only systems which meet this requirement 
are A4, B4, and C4. This is illustrated on the right-hand 
margin of Figure 4. Since, from Table 5-b, system B4 is both 
the cheapest of the relevant systems and also has the highest 
figure of merit for cost effectiveness (see Table 5-c), it is 
the logical choice in this case. 

The solution can be improved by using a modified 
approach to the upgrading of computer systems. It is gener- 
ally conceded that there will be no more revolutionary change- 
overs between generations of computers, and evolutionary 
growth will become the order of the day. With respect to 
peripheral equipment, the evolutionary approach is already 
well established and most systems are progressively upgraded 
by the addition of new peripherals, or the exchange of lower 
capacity peripheral units for those of higher capacity. This 
is especially obvious with respect to discs and other mass 
storage units. 
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Given the compatibility provided between the 
processors and operating systems within the product line 
marketed by each manufacturer, it is now feasible to apply 
the evolutionary approach to cpu's as well as to peripherals. 
It is then possible to plan the replacement of a cpu after 
one or two years' service rather than after five or six 
years. If this approach is accepted, then one no longer 
selects a single computer system but rather one selects a 
series of computers within the compatible range offered by 
one manufacturer. 

Assume that it is feasible to install a system 
for as little as one year, provided that it will be replaced 
by a compatible system from the same manufacturer's product 
line. Assume further that all installations are performed 
at the beginnings of years, and that system must be upgraded 
at the beginnings of years in which they would otherwise 
become saturated [Ein-Dor 1977] . Depending on these assump- 
tions the growth path for each of the systems would be as in 
Table 6 , and as exhibited in Figure 4. 

The discounted present value of the rental 
systems, assuming a 20 $ cost of capital, is $ 3.15 million for 
manufacturer A, $3.50 million for B, and $3.78 million for C. 
Thus the dynamic evaluation presents A as the economically 
optimal solution rather than B in the conventional evaluation. 
The total undiscounted cash flow for this solution is $ 6.65 
million compared to $ 6.98 for the static selection procedure. 
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EXAMPLES OF THE GROWTH PATHS 
FOR EACH OF THE SYSTEMS 



TABLE 6 
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The analysis above has been performed on the 
assumption that equipment is to be rented. The method is 
equally valid for purchase with purchase prices appearing in 
the analysis instead of rentals. Furthermore, if the dynamic 
analysis is performed for both lease and purchase, it can be 
of assistance in deciding on the method of acquisition. In 
using this approach for selecting the form of acquisition, 
one must take care to ensure that the values incorporated in 
the analysis are comparable. This requires that all relevant 
costs be factored in. For instance, where maintenance charges 
are included in rentals, they should either be added to pur- 
chase cost also or else rentals should be computed without 
maintenance. If purchased equipment is to be replaced, resale 
values should be determined and treated as negative costs. 

Any tax or excise differentials should also be taken into 
consideration. 

Because of varying life expectancies of units 
within the projected system growth path, and because of 
variations in the ratio of purchase prices to rentals, it is 
almost certain that an economically optimal solution will 
indicate that some units should be purchased and others 
leased. This implies that the analysis should be performed 
stepwise. As each change is made to the system in the course 
of the analysis, the profitability of lease versus purchase 
should be evaluated; new units should thereafter be consid- 
ered as acquired in the least cost mode. 
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This methodology is a basis for determining the 
relative cost-effectiveness of product lines in a given 
situation. This alone is not, of course, the only criterion 
for system selection. Other criteria such as software 
availability, manufacturer service, or reliability may well 
be at least as important. However, it is also wise to base 
one's decisions on as accurate a determination as possible 
of the cost factor. 

The steps involved in the dynamic evaluation of 
cost-effectiveness overtime for a series of computer systems 
is as follows: 

1. Determine relative capacities of relevant systems; 
possibly by benchmark runs. 

2. Forecast the workload for the planning period in terms 
of a reference system. 

3. Prepare a growth path for the systems proposed by each 
manufacturer with respect to the forecast workload and deter- 
mine rental (or purchase) costs for each system. 

4. Determine the discounted current value of each outlay 
stream, and so determine which manufacturer has the most 
cost-effective product line for the situation under study. 

The application of this method may lead to 
considerably different results than the conventional methods 
of cost comparison. It determines a unique solution for a 
unique situation which is not generally transferable. 
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f. Present Value Analysis 



A problem facing all buyers of computer equip- 



ment is deciding which system will cost the least and bene- 
fit his company the most. To determine if an investment in 
a proposed system will enhance the current value of a firm., 



present value analysis may be used. The present value of an 
amount to be received in the future is the equivalent value 
today of that future sum. 



receipts should be discounted (their face value should be 
reduced) to an equivalent present value if they are to be 
compared with present receipts. For example, the present 
value of $126.25 to be received at the end of four years has 
a present value of only $100 if the investor has an opportun- 
ity to earn six percentage on invested funds. That is, $100 
invested today at six percentage compounded annually will 
accumulate to $126.25 in four years. 

Generalizing, the present value (PV) of a future 
sum can be determined using the equation 



where FV is the future sum, K is the opportunity cost (or 
rate of return), t is the year the future sum is received or 
paid and N is the number of receipts. 

Since the decision to invest in a system should 
be considered independently of the method used to finance it. 



Because of the investment alternatives, future 
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the system should be evaluated as if it were being purchased 
for cash. The net present value (NPV) of the proposed 
system should be determined by discounting all the incremen- 
tal, after-tax cash flows associated with it, using the 
firm's cost of capital as the discount rate. The cost of 
capital is the cost of new funds required to replenish the 
cash used for the purchase of the system. This process is 
expressed by the equation 

NPV . r (R t -c t ) d- T c) t D t T c , s N -(s N -BV M ) t 0 
t+1 (l+K)° (1+KT 



where; 

R^: added gross revenue generated from the system, 

C-fc: the added cost of operating the system (operating 

costs do not include any financing cost such as interest or 
lease cost), 

T c : the firm's tax rate, 

D-f- : the depreciation, 

t: the time period, 

N: the number of years the asset will be economically 

useful, 

Sj,j: the salvage value, 

BV^: the book value, 

Iq: the initial purchase price, 

K: the firm's cost of capital. 

If the NPV is greater than zero, the system will 
add" value to the firm because its rate of return is greater 
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than the cost of funds, and therefore it is a desirable 
project. When two systems are being considered, the one with 
the larger NPV should be chosen. If two alternative systems 
provide the same service or revenue (R^), this item may be 
assigned a value of zero in the equation of NPV. In this 
case the NPV becomes negative and the system with the NPV 
closest to zero (the less costly alternative) should be 
selected . 

The relevant benefits provided by the asset are 
the net cash inflows, which are discounted at the firm's 
cost of capital to obtain their present value. If the 
present value exceeds the cost, the asset is financially 
desirable because it adds value to the firm [Roe nfelt and 
Fleck 1976, Szatrowski 1976] . 



VI. SYSTEM WORKLOAD DESCRIPTION 



The most important part of any system specification is 
the part which describes the type and amount of workload to 
be run on the system. Performance is the degree to which a 
computing system meets the expectations of the person 
involved with it [Doherty 1970J . Performance is a reaction 
of a system to a specific workload. It is, therefore, 
essential that the right workload is used when evaluating 
the system and that the workload characterization is suffi- 
ciently representative to account for all significant 
factors. A good workload description should serve three 
important functions: 

1. It should permit the vendors to determine what they 
need to propose for automatic data processing equipment and 
software (ADPE/S) to satisfy the workload requirements. 

2. It should facilitate the verification of the proposed 
systems, both as to their capabilities to handle the work- 
load, and as to the time required to complete the workload. 

3. It should permit realistic costing of the bid systems. 

The first two points, permitting determination of the 

proper system by the vendor and the verification of that 
system's capabilities, can be combined. If a good technique 
is achieved for the second purpose, that same method of 
workload description will also serve the first purpose. 
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User's applications, once translated into programs and 
commands, can be characterized by the type and the amount of 
resources the system will have to allocate to execute these 
programs and commands. The total of resource demands gener- 
ated by the user community represents the system workload 
Qvobodova 1976, Joslin 1977]. Examples of parameters used 
to describe computer system workload are presented in Table 7. 

In many computer installations, the instantaneous work- 
load changes quite unpredictably . This is especially true 
for interactive systems. The speed of the user's response 
plays an important role in what load is generated at individ- 
ual system entry points; this human factor only enhances the 
unpredictability of workload changes. It is this uncontrol- 
lable fluctuation of the system workload that makes the 
evaluation of system performance so difficult. 

Generally, the workload of a computer system has certain 
statistical properties that do not change over reasonably 
long time periods. It is then possible to: 

1. Characterize the workload by distributions of demands 
made on individual system resources. 

2. Define a unit of work and express the workload as a 
number of such units. 

Quantification of workload by work units is used when 
defining and comparing system processing capabilities. A 
unit of work is assumed to require a fixed but not necessar- 
ily explicitly known quantity of computation. Generally, 
it is very difficult to define a unit of work. Even when 
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TABLE 7 



EXAMPLES OF WORKLOAD PARAMETERS 



Workload Parameters 


Description 


Job CPU time 


Total CPU time requested by a 
single job 


Job I/O requests 


Total number of I/O operations 
requested by a single job 


CPU service time 


CPU time requested to process a 
single CPU task 


I/O service time 


I/O time required to process a 
single I/O task 


Interarrival time 


Time between two successive requests 
for a system service 


Priority 


Priority assigned to a job by the 
user 


Blocked time 


Time a job is incapable of receiving 
CPU service 


Memory requests 


Amount of memory requested by a 
single job 


Working set size 


Number of pages of a single job that 
must be kept in the main memory 


Locality of reference 


Time for which all memory references 
made by a single job remain within 
a single page or a set of pages 


User response time 


Time needed by a user at an inter- 
active terminal to generate a new 
request (think and type time) 


User intensity 


Processing time per request/user 
response time 


Number of simultaneous 
users 


Number of interactive users logged 
concurrently 


Number in the system 


Number of jobs or tasks being serviced 
or waiting in queues for system resources 


Instruction mix 


Relative frequencies of different types 
of instructions the system must execute 



Svobodova 1976 
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programs are broken down to very elementary operations (e.g., 
ins true tions ) , the characteristics of such elementary opera- 
tions differ. 

A workload model serves as a workload of a real computer 
system during performance measurement experiments or as an 
input to a model of the evaluated system. The purpose of 
using workload models is to: 

1 . Provide representative workloads for comparative 
performance evaluation of different systems. 

2. Provide a controllable environment for experimental 
performance optimization studies. 

3 . Reduce the quantity of data that have to be analyzed. 

4. Present the system workload in a form required by a 
system model. 

Alternate choices in system configuration and algorithms 
and the effect of different control parameters must be evalu- 
ated for the same workload. Generally, only one alternative 
can be examined at a time, thus requiring that the workload 
used as the input to the system during evaluation be 
reproducible . 

The characterization of workload by demands made on 
system resources can also be used to define a unit of w ork. 

In fact, the workload parameters given in Table 7 are already 
defined with respect to a specific logical unit processed by 
a computer system. Such a logical unit is often adopted as 
a unit of work. To satisfy the requirement that a unit of 
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work represents a fixed quantity of computation, this logical 
unit is afixed with characteristics representing the mean of 
characteristics of all such units processed by the system. 

The real system workload, that is, the workload generated 
by the user community in the normal production environment, 
is generally not reproducible in its exact composition. 

However, if the statistical properties of the system work- 
load do not change with time, the workload is statistically 
reproducible. The real workload can be used to drive the 
system during evaluation experiments, but the measurement 
intervals must be sufficiently long, and it is necessary to 
collect and analyze large amounts of data to ensure that the 
statistics are correct. The minimum measurement interval 
may range from minutes or hours if the system workload does 
not change with the time of day, to weeks or months if the 
workload exhibits significant changes with such periodicity. 

System workload may remain stationary for quite long 
periods of time, but in general, its characteristics change 
slowly as the user community changes because new applica- 
tions are added and the old discontinued. In addition, the 
user community tends to adapt to system changes, and as the 
users change their habits, workload characteristics change. 

Thus in a long term, the real workload is not reproducible. 

System workload is characterized by demands for system 
resources. Ideally, a workload model will have the same 
characteristics as the real workload. The model is accepted 
as being representative of the real workload if its application 
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results in the sane steady state performance ^errari 1972 , 
Svobodova 197 . The lack of proper understanding of work- 
load characteristics is a serious obstacle when the goal is 
to predict effects of system changes and design alternatives 
on performance. Extensive empirical studies of programs may 
reveal many interesting properties that should be considered 
during the initial system design. It is also important to 
study the habits of system users. Performance effects of 
certain system changes measured against a once representative 
workload model may be positive, yet in reality, the users may 
react to these changes in such a way that the overall effect 
will be negative. The true behavior of the eventual users 
may be quite different from the behavior assumed for the 
purpose of system selection or design pia rner 19727 . 

A system that is too carefully tuned to a specific 
projected workload might not meet the performance objectives 
if the real workload turns out to have different character- 
istics. It is thus necessary to have a means of examining 
performance in the light of different workloads. Flexibility 
and controllability of workload characteristics is an important 
property of a workload model. 

A. INSTRUCTION MIX 

An instruction mix represents the relative frequencies of 
different types of instructions a system must execute during a 
specified interval of time. The instruction mix specifies rela- 
tive usage of different types of instructions in a particular 
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application. Since each instruction may require a different 
time to execute, performance (instruction execution rate) of 
an instruction set processor can be evaluated with respect 
to the requested instruction mix. Instruction mix is used 
in two main areas: 

1. Selection of computer hardware, 

2. Design of new processors. 

In the first case, the typical instruction mix for the 
class of applications planned for the system must be defined 
such that it can be used across a wide range of different 
instruction sets. That is, a typical instruction mix speci- 
fies frequencies of different functions, rather than actual 
instructions that perform these functions. A typical mix 
might be in the proportion of five adds, two compares, one 
subtract, one multiply. This, then, might be described as a 
mix of instructions. By multiplying the frequency for which 
the instruction is typically used by the time a particular 
machine takes to perform the instruction and adding these 
together for the instruction mix, one can arrive at a figure 
which represents the time for the instruction mix on the 
particular machine. This figure can be calculated for vari- 
ous machines and thereby the machines compared for an instruc- 
tion mix appropriate to a particular type of job. These 
figures can be arrived at from the theoretical timings for 
instructions of the machine or they can actually be measured 
by running a mix containing the appropriate number of 
instructions on the machine. 
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As instruction times are in milliseconds or microseconds , 
it is convenient to run the mix, maybe., ten thousand times 
in loop consecutively so that the start and finish can be 
measured in minutes on a stop watch. Some of these mixes 
have been commonly adopted as means of comparison. The most 
frequently used instruction mix is the Gibson mix., which can 
be classified as a "general purpose" mix. Figure 5 shows 
the calculations of the mix time and Figure 6 shows the 
approximate Gibson mix times in milliseconds for a number of 
ma chines . 

Instruction mixes are also a means of comparing the speed 
of doing arithmetic in machines. Even so, this simple form of 
comparison may be prejudiced by dissimilarities between hard- 
ware which, perhaps, are advantageous to one machine and not 
the other. For example, word length varies between machines 
and to the user this is of some importance from the point of 
view of accuracy; i.e., in a process control application it 
may be perfectly adequate for the hardware to handle only 
four decimal digits, however for numerical analysis applications 
ten decimal digits may be required. Quite obviously a simple 
comparison of arithmetic operation is valuable only with other 
information about the specific applications [Graham and Yearsley 
1973* Svobodova 1976, Gibson 1970, Joslin 1977^ . 

Instruction mix depends on many factors that are diffi- 
cult to account for, such as the number of operands per 
instruction or different addressing modes. Due to these 
factors, the number of instructions needed to run the same 
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FIGURE 5 



CALCULATION OF MIX TIME 



INSTRUCTION 


MILLISECONDS 


SECONDS 


5 adds 


10 


50 


2 compares 


5 


10 


1 multiply 


10 


10 


1 subtract 


50 


50 


MIX TIME 




0.120 





FIGURE 6 


APPROXIMATE 


PERFORMANCE FIGURES 


PROCESSORS 


GIBSON MIX MILLISECONDS 


ATLAS 


0.23 


7094 


0.28 


1106 


0.24 


360/65 


0.17 


1108 


0.11 


360/75 


0.11 


1906A 


0.10 


6600 


0.09 


1110 


0.025 


370/195 


0.02 


7600 


0.01 


370/165 


Approx. 0.07 


370/1^5 


Approx. 0.15 


370/1&5 


Approx. 0.30 



Yearsley 1973 
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task cn different machines may vary significantly. Also the 
instruction mix is dependent on the programming language in 
which the application is coded, the translator of this 
language, and finally the programmer [£unde 197 ^, Svobodova 

19767 • 

B. BENCHMARK PROGRAMS 

A benchmark is defined as "a point of reference from 
which measurements can be made" (Sippl 1972] . A benchmark 
can be an instruction, a special program or a sequence of 
calls to selected software components. In most cases, 
however, the term benchmark is used to mean a job or a sec 
of jobs that represent a typical workload of the evaluated 
system. Benchmarks play the role of a drive workload in the 
real system, both for the purpose of comparative evaluation 
of different systems and performance optimization. A good 
benchmark will exercise all system functions (job scheduling, 
file management, I/O support, language processor, etc.) in a 
manner in which these functions are used or are expected to 
be used in the actual production environment. 

A benchmark representative of the current system workload 
can be assembled from already existing programs. Jobs to be 
included in the benchmark may be selected by random sampling 
of the job stream. This method does net require an explicit 
knowledge of characteristics of individual jobs, but it is 
then difficult to determine how many of these randomly- 
selected jobs must be included in the benchmarks [Shope 197 o] . 
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The real system workload generally consists of several 
classes of applications (scientific problems, payroll, file 
update, etc.). A benchmark or, as it is sometimes called, a 
benchmark mix, can be constructed as a properly weighted mix 
of jobs representative of each class. However, demand 
characteristics of jobs performing different functions may 
greatly overlap. 

The most rigorous approach rests cn partitioning jobs 
into classes according to their characteristics. The job 
with characteristics closest to the typical characteristics 
for its class is selected to represent the class in the 
benchmark. A selected job is assigned weights proportional 
to the percentage of workload that falls into that same 
category. Partitioning of jobs according to their true 
characteristics can be accomplished by cluster analysis. A 
clustering algorithm assigns jobs to a predetermined number 
of groups called clusters such that the differences between 
members of the same cluster are small compared to differences 
between numbers of different clusters. 

A benchmark constructed from real jobs is apt to be 
system dependent. In general, such benchmark is not directly 
usable as a drive workload of a different system. A consid- 
erable conversion effort may be necessary to create a bench- 
mark for several different systems Qoslin 1977* Svobodova 
1976, Joslin 1965 ., Hosen 1976, Hunt, Diehr and Garnatz 197*3 • 
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There are several steps to obtaining the mix of represen- 
tative programs to be used for benchmarking purposes. The 
following general guidelines should be kept in mind while 
searching out representative benchmark programs: 

1. Where possible, benchmark programs should be written 
in a standard higher-level programming language; e.g., ANSI 
FORTRAN or ANSI COBOL. 

2. The mix of benchmark problems should be small enough 
that it is capable of being processed during a single half- 
day benchmark demonstration. 

3. The selected mix of benchmarks will demonstrate that 
the supplier's proposed system contains adequate memory and 
input/output devices, that the software proposed is opera- 
tive and adequate, and that it has sufficient throughput 
speeds to the normal workload. 

The benchmark programs are not to be selected to prove 
the worst case situation, but rather to demonstrate timing 
and capability for normal situation. If it is necessary to 
assure capability to handle worst case situations, benchmark 
programs selected for that purpose will be obtained; but 
they are not to be included in the mix; rather they will be 
treated separately as capability benchmarks. 

The results of the benchmark will help in the evaluation 
effort by: 

1. Proving that the vendor has a deliverable compiler and 
operating system. 
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2. Allowing the evaluator an opportunity to compare the 
relative speeds of the different compilers. 

3. Determining the relative efficiency of the generated 
object code by comparing results of execution of the compet- 
ing object programs. 

4. Giving the evaluator sufficient information to allow 
determination of minimum internal memory requirements. 

This last calculation is based on the size of the 
largest program which must be memory-contained at any one 
time, and is derived from: 

1. The benchmark results, which allow determination of the 
ratio of object language instructions to Procedure Oriented 
Language (POL) statements, and the average size in terms of 
memory locations of the object instructions. 

2. A user-generated estimate (in POL statements) of the 

size of the largest program to be core-contained. It is 
necessary to calculate the memory required for the program 
(MP) from the formula: M? = R x NS x IS, where: 

R = ratio of object language to POL statements (from the 
benchmark) , 

NS = (estimated) number of POL statements for the largest 
program, and 

IS = average instruction size determined from the benchmark 
by dividing the memory required by the number of object 
instructions , 

The estimate of required internal memory is completed by 
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adding the program requirements, the memory requirements for 
the operating system resident, data and buffer storage, and 
communica tions-oriented subroutines . 

3. Simple multiplication of the following factors: 



benchmark object code 
benchmark POL 



. . . . estimated object 

x estimated POL = code 0 



4. An estimate of the size of the average object instruc- 
tion. This is obtained from the benchmark by dividing its 
memory need by the total number of object language statements. 

3. Estimate of core for resident supervisor input, output 
buffers, and communica tions-criented subroutines. 

The summation of 4 and 5 will give an estimate of the 
required internal memory [Rubin 197lJ . 

1 . Derivation of Representative Programs 

The following paragraphs describe a method for 
obtaining the representative programs. 

a. Application 

List each of the applications making up the 
total workload. This is illustrated in Table 8. 

b. Programs and Tasks 

For each computer program pertaining to the 
above applications, list the program and provide the infor- 
mation required in Table 8. For new programs or for 
acquisition of equipment that is for a new installation, it 
will be necessary to go through the normal design process 
with program flowcharts which lead to estimates of program 
run times, or to simulate the programs to obtain this infor- 
mation. Once the necessary estimates are obtained for 
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TABLE 8 





APPLICATION, PROGRAM AND TASK 


INFORMATION 




Aoolication Individual 




Estirna ted 




and 


Program Run Time 


Frequency 


Monthly 




Tasks (hours) 


Per Month 


Time (hrs) 


Subtota 1 




(1) 


(2) 1 


(3)- (l)x(2) 


(4) 


Application A 








ADMINISTRATION 








Al: 


Time /Cost 










Studies 0.52 


22.0 




11.44 


a . 


Sort 




10.00 




b . 


Edit 




1.44 




A2 : 


Payroll 










Processing 0.25 


4.3 




1.08 


a • 


Sort 




1.00 




b. 


Va lida te 




0.08 




A3 = 


Transports tion 










Usage 






9.30 


a . 


Compile - 










Cobol 0.50 


1.0 


0.50 




b. 


Extract 0.4C 


22.0 


8.80 




A 27 


: Miscellaneous 






20.00 


a . 


Sort 




10.00 




b. 


Upda te 




10.00 




Application B 








DISTRIBUTION CONTROL 








31: 


Inventory 










Control 0.45 


22.0 




10.00 


a . 


Sort 




10.00 




B2 : 


Allowance 










Genera tion 






2.50 


a . 


Compile - 










Cobol 3.00 


1/6 


0.50 




b. 


Extract 12.00 


1/6 


1.00 




c . 


Compute 




1.00 




Appl 


ication K 








MATRIX INVERSION 








Time 


Required on System 




Total 


880.00 



Joslin 1377 
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Table 8,’ these new programs can be treated in the same way 
as any existing programs. Each program is broken down into 
its major functions or tasks, such as, sort, validate, update, 
extract, compute, card-to-tape conversion, tape-to-printer 
conversion, trajectory calculation, simulation, matrix 
manipulation, etc. 

c. Task Summary 

From each of the programs listed in Table 8 
extract similar tasks and prepare a Task Summary Sheet for 
each task (see Table 9 ). Provide the information required 
in accordance with table headings which are explained below, 
which is the description of the columns in the Task Summary 
Sheet. 

IDENTIFICATION. This column contains the code 
for each program in which the task is found. In the example 
3hown in Table 8 the identification codes which would be 
given on a Sort Task Summary Sheet would be Ala, A2a,...A27a 
and Bla, etc. 



I/O o 


r FILE 


DESCRIPTION. This section is divided 


into four parts: 






1. Media Code. 


Enter 


a mnemonic for the media that 


contains the I/O 


or file 


Examples are: 




MT 


Magnetic Tape 




PT 


Paper Tape 




PC 


Punched Cards 




PR 


Printer 
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TABLE 9 



TASK SUMMARY SHEET - DESIGN 



The column headings are successively: 
Identification 
I/O or file description 
Media code 
Number of devices 
Ca tegory 
Block size 
Monthly averages 
Frequency 
Volume 

Total time (hours) 

Peripheral equipment time 
Magnetic tape 
Card 
Printer 
Other 

Internal storage requirements (in K's of storage) 
Language 

Present 

Planned 
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2. Number of Devices. Number of devices that will be 
required for the use of this media which will have the same 
capability and category of use. 

3. Category. Code designating the type or use of the 
I/O or file. The following codes shall be used: 

0 Source or original input 

1 Master file 

2 Intermediate, working or scratch 

3 Final output 

4. Block Size. Product of the number of characters per 
record and records per block. 

MONTHLY. This column is devided into four parts: 

1. Frequency. Give the monthly run frequency of this 
program. 

2. Volume. The number of blocks contained in the I/O or 
file. If this is a multi-tape file, follow the number of 
blocks by a slash and give the number of tapes. The volume 
to be recorded will be the average per unit, per month, for 
this task. 

3. Total Time. Average total time to perform this task 

in the identified program. All times shall be given in hours 
and hundredths of hours. 

4. Peripheral Equipment Time. Estimated average time 
required per task by each type of peripheral equipment. If 
similar units of differing capability are used, this timing 
information should be based on the highest capability avail- 
able. Due to simultaneity and overlap, it is not expected 
that the total of the individual units will agree with the 
total time. 
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INTERNAL STORAGE. Estimated amount of internal 
storage required to process the task (not the full program, 
if the program is a multi-task program). If two or more 
processors are used for the task, enter the information 
accordingly . 

LANGUAGE. List the language in which the task 
is programmed (first column) or is to be programmed (second 
column). If task is a self-contained library routine, the 
initials "L.R." should be entered. 

TOTAL TASK TIME. At the end of the last Task 
Summary Sheet used for each type of task, there should be a 
line for total task time. The sum of these totals from all 
the Task Summary Sheets should equal total system time. 

TYPICAL TASK. On the last entry of last Task 
Summary Sheet used on each type of task, there should be an 
entry for a nonexisting program. This entry should be 
weighted average (weighted by used time per month) for all 
previous entries for this type of task, and it should depict 
what a typical task of this type would look like, 
d. Selection of Representative Tasks 

From each of the sets of tasks, select tasks 
(preferably a single task program) which are representative 
of the set, or a substantial portion thereof, and identify 
these tasks with asterisks. The types and time of proces- 
sing, amount of internal storage used, language used, and 
equipment configuration should all be taken into account when 
selecting the representative task; that is, it should be as 
nearly similar to the typical task as possible. 
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Extension Factors 



e . 

A chart should now be prepared showing each of 
the representative tasks and the functions each presents. The 
monthly times required for each of these functions within a 
task should be listed alongside of the individual times of 
the benchmarks chosen to represent these task functions. The 
individual benchmark times for the functions should be di- 
vided into monthly times for these functions to obtain indi- 
vidual extension factors, which show how many times a month 
the representative benchmark would have to be run to make up 
the full monthly workload for the task. An example is shown 
in Table 10. If only sequential systems were to be consid- 
ered, the individual functional extension factors would be 
sufficient, and each vendor could run the benchmark program, 
extend his system running time for each benchmark by the 
functional extension factors just derived, and thus be able 
to tell how long his system would take to complete the total 
workload . 

In third-generation systems where many degrees of 
simultaneity must be considered, it is possible that while 
the system is handling one function, it could simultaneously 
be handling another, or even be multiply handling various 
tasks on programs. Thus, all the representative programs 
must be considered together to form the representative sample 
of the total workload. If the normal workload could be 
processed in variable ways, then a vendor should be permitted 
to handle the grouping or mix of representative benchmark 
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TABLE 10 



REPRESENTATIVE PROGRAMS 



Time (hours) 



Task Set 


Workload 

Functions 


Monthly 

Task 


Represents - 
tive Task 


Extension 
Fa ctor 


Sort 


Total thruput 


145. CC 


(single run) 
0.45 


322 


B-la 


Ma g . tape 


125.00 


0.25 


500 




Card reader 


115.00 


0.03 


3833 


Edit 


Total thruput 


120.00 


0.75 


160 


2 -4a 


Ma g . tape 


80.00 


0.60 


133 




Card reader 


20.00 


0.50 


40 




Printer 


ICO. 00 


0.25 


400 


Update 


Total thruput 


100.00 


0.16 


625 


D-5a 


Mag. tape 


70.00 


C.10 


700 




Card reader 


25.00 


0.05 


500 




Printer 


50 . CO 


0.10 


500 


Matrix 


Total thruput 


90.00 


0.45 


200 


Inversion 


Card reader 


3.50 


0.02 


175 


K-6a 


Mag. drum 


24.00 


0.15 


160 




Printer 


1.50 


0.01 


150 


FORTRAN 


Total thruput 


85.00 


0.17 


500 


Compile 


Ma g . ta pe 


78.00 


0.15 


520 


H-3a 


Card reader 


6.00 


0.02 


300 




Printer 


4.00 


0.01 


400 


COBOL 


Total thruput 


40.00 


0.12 


333 


Compile 


Mag. tape 


38.00 


0.11 


345 


G-2 


Card reader 


4.00 


0.04 


100 




Printer 


3.00 


0.04 


75 


Tape to 


Total thruput 


300.00 


1.00 


300 


Print 


Ma g . ta pe 


300.00 


1.00 


300 


P-4 


Printer 


300.00 


1.00 


300 



Total Monthly Time 880.00 



Joslin 1977 
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programs in whichever v; ay his system can best handle them. 

If normally one type of workload would be handled before the 
other, then the mix should be structured in that way. How- 
ever, just taking one each of the representative benchmark 
programs does not make a representative mix because the 
related extension factors also have to be considered. Table 
11 shows a mix of representative benchmark programs and the 
mix extension factor. 

f. Mix of Tasks 

The extension factor for the mix is derived by 
examining the information contained in Table 10 and obtain- 
ing the lowest practical extension factor to reduce the 
number of problems to be run in the mix while retaining the 
required representative nature of the mix of problems, which 
in this case is l60. This extension factor is then divided 
into each of the sequential extension factors to obtain the 
quantity column. The previous column is then used to make 
the input/output total time when extended by the mix exten- 
sion factor equal to the total projected input/output time. 
This mix of tasks can then be used as a proper demonstration 
of a supplier's multiprogramming or multiprocessing [joslin 
1977, Stimler 1974J . 

2 . Expected Workload Levels 

The workload to be processed by a system can be 
expected to increase over time. Therefore, the workload for 
a system can be envisioned as consisting of a series of 
various levels. These workload levels can be roughly 
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TABLE 11 



A MIX 0? REPRESENTATIVE 
BENCHMARK PROGRAMS 



PROGRAM 


QUANTITY 


PROVISIONS 


B-la 


2 


Norma 1 


E-4a 


1 


Input from tape 


D-5a 


4 


Twice normal 






Twice no output 


K-6a 


1 


Norma 1 


H-3a 


3 


Norma 1 


G-2 


o 


Norma 1 


F-4 


2 


Norma 1 



Extension Factor for Mix: 160 
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approximated by the average monthly workload levels for each 
year of system life. The workload increases from one level 
to another as the system ages increase. Because of the 
uncertainties that are associated with projecting workload 
growth overtime, it is impossible to predict with complete 
accuracy just when the workload will reach a given level. 
Therefore, probabilities are used for this purpose as 
described below: 

a . System Life 

A chart showing system life should be prepared 
showing the number of years that the system is expected to 
be in existence (see Figure 7-a). 

b. Projected Growth 

A best-guess approximation of projected growth 
of the system should then be superimposed on the foregoing 
chart, the vertical axis depicting the workload in hours-per- 
month. Figure 7-b shows this. 

c. Workload Levels 

At the midpoint of the projected growth line for 
each year, construct a workload level line parallel to the 
horizontal axis (see Figure 7-c). 

d. Level Probability 

For each year of system life enter the probability 
of the average workload for that year being at or near each 
of these levels. The probabilities must be thought of as 
lumped at these levels in such a way that the total probabil- 
ity for a year adds up to 100$, because the number of levels 
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used is finite Q'oslin 1977; Svobodova 1976). For example, 
referring to Figure 7-d, it can be seen that the workload 
there illustrated has a probability of 90 ^ of being at level 
one for the first twelve months and a 5 % chance of still 
being there for the second twelve months. Therefore, each 
vendor would be asked to determine the configuration 
necessary to process workload level one in the allotted time 
period, and to determine the monthly cost of that configuration. 
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FIGURE 7-d 



EXAMPLES OF LEVEL 
PROBABILITIES 



Level : 
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0 


0 
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2 
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5# 
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0 


1 
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5# 
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3rd 


4th 


5 th 
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VII. METHODS OF PROCURING COMPUTER SYSTEMS 



The three most commonly used computer procurement plans 
offered by the vendors are lease, purchase, and lease with 
option to purchase. The decision on the selection of 
computer hardware (and software) and procurement methodology 
is a management responsibility, and should be based upon a 
feasibility study and subsequent evaluation process. In 
connection with hardv/are selection, the manager always makes 
a second decision; that is the decision as to whether to 
purchase the computer or to rent it. The feasibility study 
should include recommendations on the purchase-rent decisions 
and the facts upon which these recommendations were based. 

A. COMMON PROCUREMENT PLANS 

In this section the common methods of procuring computer 
systems will be introduced. 

1 . Leasing 

Leasing, in the context of computer use, usually 
means an operating lease, with ownership of the computer 
system retained by the vendor. The user pays a predetermined 
monthly price for the use of a certain length of time on the 
computer system. The lease price includes rental of the equip- 
ment, a fee for the maintenance and service of the equipment, 
and a payment to compensate the vendor for the risk of owner- 
ship. The length of the lease is very important in its effect 
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on the rate. Since the leases which are proposed by the 
manufacturers are usually short-term ones, the following 
discussion will concern itself with these short-term leases. 

Under a normal short-term operating lease, the user 
enjoys the following advantages: 

1. Frees working capital for more productive use (since 
money is not tied up in low-yielding fixed assets). 

2. May cost less than other methods of acquiring equipment. 

3. May increase the firm's ability to acquire funds. 

4. Establishes only a restricted (not a general) obligation 
against the company which may be satisfied by payment of one 
year's rent in bankruptcy or three years' rent in reorgani- 
zation. 

5. Does not appear as a liability on the leasee's balance 
sheet. 

6. Leaves normal lines of bank credit undisturbed. 

7. Permits 100$ financing (as against 75$ or 80$ through 
other methods). 

8. Creates an allowable cost (or acceptable cost according 
to the government regulations including interest cost) under 
government contracts. 

9. Permits hedging of business risk (primarily the risk 
of obsolescence). 

10. Minimizes danger of being oversold. 

11. Assures more adequate servicing (since maintenance is 
the responsibility of the lessor and usually included in the 
lease contract). 
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12. Offers the convenience of making only one payment 
(rather than separate payments for debt service, maintenance 
cost, insurance, property taxes, etc.). 

13 . May be tailored to the leasee’s computer system needs 
more easily than ordinary financing. 

14. Avoids che necessity of selling equipment no longer 
wanted. 

15. Permits middle-management executives to acquire new 
equipment without going through formal appropriation request 
procedures . 

16. Provides cost-cutting equipment to be installed 
immedia tely . 

17. Acts as a hedge against inflation. 

J 18. Provides long-term financing without diluting owner- 
ship or control. 

Prom the point of view of the leasee, equipment leasing has 
the following disadvantages: 

1. Equipment leasing charges a higher interest rate (than 
the leasee's regular interest rate). 

2. May provide less attractive tax deductions (than inter- 
est plus accelerated depreciation). 

3. Gives any residual value of the equipment to the lessor. 

4. Establishes a fixed obligation against the company. 

5. Does not provide whatever prestige that goes along with 
ownership. 

6. Raises the fear of dispossession if payments are not 
made during hard times. 
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The importance of the listed advantages depends on 



the individual company and the environment in which the com- 
puter is to be used. System obsolescence within the organi- 
zation that is using the computer has proved much more 
significant than technological obsolescence. The knowledgeable 
prospective user, in forecasting the life expectancy of the 
proposed system, should carefully study and evaluate his data- 
prccessing needs. However, since every user does not exercise 
the same degree of foresight in planning, the vendor must set 
his lease charges so that they allow for the average system 
life expectancy. This is a compromise measure since the 
vendor must deal both with those users who plan and those who 
do not. One can therefore see that if the user has done a 
good job of planning his systems, he is in a better position 
to assume any risk of system obsolescence than the manufac- 
turer is. Also, it can be costly for the user to lean on 
obsolescence as a crutch or to allow it to influence the 
lease/purchase decision. 

Another important disadvantage of leasing is that on 
a leased computer system, extra usage is more expensive than 
it is on an owned system. And if the user makes any serious 
attempt to make the computer pay for itself, he may have to 
utilize these extra shift hours [Joslin 1977> Vancil 1962, 
Gustafson 1973]. 

The concept of the third-party operating lease on 
computer equipment originated in the United States. Indeed, 
even in Europe, the service has been provided almost 
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exclusively by US-based companies. Its origin can be traced 
as far back as 1961 when D. P. Boothe Inc. wrote an operating 
lease on an IBM 7094 for Ling-Temco-Vought . The principle 
attraction to the user was a saving in additional use charges, 
then 40^ of the primary shift rental. 

Second -genera tion computers were not really amenable 
to leasing because any significant increase in power or 
capacity required a change of hardware, sc that even a 
medium-term commitment would not have been tolerable. This 
constraint disappeared with third-generation systems which 
permit considerable growth through addition rather than 
change. At the same time, the total cost of running a com- 
puter was steadily mounting; because of the increased power 
and sophistication of the equipment, correspondingly more had 
to be spent on software and other supporting functions. The 
overall cost meant that the user had to think in terms of a 
longer life span for each system he installed, in the range 
of say three to five years {Gustafson 1973* Szatrowski 197 6 j . 

The hardware itself is now sufficiently reliable, 
flexible and modular for a functional life span of at least 
ten years to be foreseen, against four years for rental to 
become equivalent to the purchase price. Being prepared to 
wait longer than this to recover their costs, the leasing 
companies could provide the equipment at less than the manu- 
facturer's rental. 

The subject of computer leasing revolves almost 
entirely around IBM computers. Because of IBM's large market 
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share, early in the game the computer lessors elected to 
purchase IBM equipment since they felt it had the best chance 
of being placed elsewhere. A company with a small market 
share' would create unacceptable risks to the lessor. The 
initial success of computer lessors so far as their ability 
to raise large quantities of capital and to convince many 
users of the viability of the concept is well known. But 
changes in IBM policy curtailed their growth. 

For many years IBM was a one-price shop. That is to 
say, it made no difference whether you used one computer or 
ten, your unit price was the same. It made no difference 
whether you could use the equipment for five years, ten years, 
or one year. Your price was the same. It made no difference 
that technical requirements that you could project did not 
indicate or consider significant growth. Accordingly, many 
users were subsidizing the requirements of the more sophisti- 
cated user. Computer leasing then became a viable alternative 
because the computer lessors were able to offer leasing pro- 
grams that more closely matched a customer's requirements. 

The discount offered by the computer lessors is the 
obvious advantage to the user as compared with leasing from 
the manufacturer. Services provided by computer lessors vary 
substantially from company to company. Some provide services 
on the theory that they will enhance the ability to move 
equipment around. Others got into other services for diver- 
sification reasons with the expectation that they would be 
getting into other profitable businesses. For example, at 
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one time Randolph Computer Corporation owned a number of data 
processing centers in the Pacific Northwest and in the Mid- 
west. These provided a whole range of services normally 
found in such centers but really had no direct relationship 
to the computer leasing activity. It gave a reservoir of 
skills in programming and systems engineering that could be 
used from time to time in the leasing activity. However, 
only very rarely were these skills found to be required. 

As time propressed, computer lessors have recognized 
that they cannot look at themselves exclusively as offering 
a financial service, that is to say, renting a computer at 
a price that is less than the user would pay IBM. The busi- 
ness has become increasingly technical in nature. It is 
obviously beneficial if it can be demonstrated to a user that 
there are more effective equipment configurations to meet his 
requirements... This is advice that he will not always get 
from the manufacturer since it sometimes implies less 
equipment [jearsley 1973> Gustafson 1973.. Randolph 197^J . 

It can be told to the customer how efficiently his 
equipment is being used by the use of a hardware monitor. 

Then the customers must frequently be assisted in upgrading 
from one model to another. For example, upgrading from a 360 
model 20 to a model 30 has many software ramifications and 
requires a considerable amount of handholding. 

There are some risks to a company doing business with 
a computer lessor. As with maintenance, in the early days of 
computer leasing there was some fear on the part of the user 
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that by cutting the umbilical cord to IBM, he might be cut- 
ting himself off from valuable services. It is alleged that 
there have been instances where the salesman on an account 
may have implied that this was so. This was basically 
contrary to the ground rules under which IBM is supposed to 
operate. Fortunately, these instances have not been fre- 
quent. The more important risks lie in the area of whom one 
chooses to do business with. Some of the computer lessors 
have gotten themselves into financial difficulty as a result of 
unwise diversification efforts. Others have withdrawn from 
the computer leasing field because they find that their 
diversification efforts have been so successful that they 
elected to concentrate on them rather than on computer 
leasing. Customers doing business with such companies 
obviously do so at their peril. 

It is important that a computer lessor be chosen for 
its financial stability and its demonstrated ability to stay 
in the business for the long haul. Flexibility that customers 
require can be met only by someone who is wholly committed 
to this business {^ustafson 1S73> Sabol 1972] . 

It is also important that the equipment be maintained 
in the best condition. Routine maintenance is, of course, an 
important part of accomplishing this. In addition, when 
equipment is moved from one customer to another, it frequently 
goes through a refurbishing center where catch-up maintenance 
is performed. The pressures of day-to-day work at an instal- 
lation often will not permit all the maintenance routines to 
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have been effectively completed. This can be accomplished at 
a refurbishing center. It is also important that installation 
be performed smoothly and with a minimum of disruption. Proper 
preinstallation planning is an essential ingredient. One of 
the measures of the effectiveness of any computer leasing 
organization is how well it contends with emergency situa- 
tions as they occur [Randolph 1974; Oliver 1973*] . 
a. Lessee Motivation 

However mixed the options of the manufacturers, 
however varied the fortunes of the leasing companies, the 
computer user who employed a leasing facility has generally 
found it highly rewarding. Without any significant change in 
his relationship with the supplier, the user has been able to 
obtain exactly the same equipment for between 10 and 25^ less 
than the normal rental charge graham 1973 } • 

Substantial cost savings have, therefore, been 
the principle benefit to the user. Also, leasing offers 
another option in the choice of acquisition method. Tradi- 
tionally the manufacturer offered two alternatives: rental, 

usually for a minimum of twelve months only, or outright 
purchase. Leasing provides a further choice: a lower 

rental charge for a longer period of commitment. The 
previous rental user and the previous purchase user both 
found the leasing proposition attractive. This is because 
the conventional alternatives are best only in extreme situa- 
tions which are not the normal user requirements: short-term 
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rental Is suited to the user who wishes to make frequent 
major changes to his equipment, and purchase to the very 
stable environment where a commitment of six years or more 
is acceptable and where, also, capital and credit are readily 
available and not better employed for other purposes. 

In practice, even for the rental user, major 
equipment changes cannot be economically made in less than 
two to three years so that the flexibility provided with a 
twelve-month agreement is more illusory than real; it is in 
fact a relic of the punched card era when change was less 
pervasive in its effect on a company's overall activities. 

The purchase user already realised that the 
manufacturer's rental terms offered flexibility he did not 
need at a price he did not want to pay. However, the purchase 
alternative contained two deterrents: first, commitment to 

hardware over a period long into the future (at least six 
years) during which unforeseen requirements could arise for 
computer processing power, and during which technological 
advance could make the equipment obsolete; and second, tying 
up large amounts of capital or credit which could normally 
be employed better elsewhere in the company, in its own line 
of business. 

Leasing, therefore, provided a very acceptable 
compromise between the extremes of rental and purchase, and 
the commitment of two to five years, tailored to the user’s 
plans, was less of a hardship than a correlation with real 
requirements. Furthermore, different components of the 
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system could be rented or purchased if these methods of 
acquisition were selectively found most suitable. For example, 
if a faster printer is planned one year after the main instal- 
lation, this could be rented direct from the manufacturer, 
while the rest of the system is leased. 

In addition to the saving against the manufac- 
turer's normal rental charge, leasing contains a further 
tangible advantage. The leasing company's charge normally 
covers use of the equipment 24 hours a day, whereas the 
manufacturer's standard rental is for a specified number of 
hours per month, roughly equivalent to a single shift five 
days a week. An additional charge is made for use beyond 
this period. It is true that the leasing customer will 
probably incur an additional charge for maintenance, but 
this again is less than the rental alternative [Graham 1973> 
Coutinho 1977* Tatham 1969 J. 

There are a variety of ways in which the leasing 
customer can capitalize on the benefits available. Most 
obviously, he can reduce the cost of his computer installa- 
tion. Alternatively, he can have more equipment or more 
people for the same expenditure as previously budgeted under 
the manufacturer's rental plan. In addition, there are more 
far-reaching opportunities for the user, the benefits of which 
could far outweigh the direct savings. Through leasing, it 
may well be possible to install a system of greater power than 
originally envisaged, and then keep it for longer. A leased 
360/40, for example, may cost as little as a 360/30 in about 
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two years. The user who needs the power of a Model 30 now 
and a Model 40 in. say, two years' time could afford to 
install the Model 40 at the outset and then hold it for five 



in acquiring more power for the same money, but rather in 
conferring stability on the data processing departments. By 
avoiding frequent changes in hardware the user avoids the 
concomitant expense of software and procedural changes. 

Where there are frequent major changes in equipment, the 
energies and costs of the data processing department are 
expended on technical transition from one language to another 
or from one operating system to another. This does not make 
profit for the company. However, by first establishing a 
stable technical environment for, say, five years, the data 
processing staff can then concentrate on its main purpose of 
increasing the computer's functional contribution to the 
company's business. In this way, the leasing facility pro- 
vides the opportunity not only for better value from expen- 
diture on the hardware itself, but for better value from the 
whole computer investment. 



customers of the leasing companies for a number of reasons: 
they have more to gain in absolute terms, they were quick to 
perceive the advantages, and they were attracted to the 
leasing companies because of their credit standing and 
because they tended to have medium to large computer systems 
which most lessees favored [Randolph 1976, Graham 1973] . 




The real advantage from doing this is not only 



Major companies tend to predominate among the 
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b. Lessor Motivation 

Like the lessee, the lessor is in the business 
for the money he can make out of it. Already a number of 
entrepreneurial fortunes have been made (and some lost) in 
the USA from this business and some companies of substance 
have emerged, already diversified into other fields. 

The lessor's starting point is his willingness to 
take an eight- to ten-year view of the computer as a revenue- 
earning investment. He supports this position in a number of 
ways: first, the computer is an electronic device with little 

to wear out; second., third-generation computers are suffi- 
ciently reliable and modular to have a long working life; 
third, they have proven to be compatible with the next 
generation of hardware; fourth, the pace of technical change 
ir. the computer industry is slowing down. 

The lessor's view of the machine is thus quite 
different from the user's. It is also quite different from 
that of the manufacturer's. In developing and building a 
computer, or family of computers, the manufacturer has 
invested huge sums of money. Even before the first computer 
of a new 'generation' reaches its first customer the manu- 
facturer has spent millions of dollars on research and 
development, and on plant and equipment. He has to recoup 
this within as short a period as the market place will allow, 
in a market place which is rental-oriented. In practice the 
selling price of a computer is normally recovered in approxi- 
mately four years of rental (jSucci 1973 , Borovits 1975 ]. 
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So the manufacturer's rental-to-purchase ratio is 
four years, the user thinks in terms of three to five years, 
and the lessor believes it will earn reasonable revenues for 
eight to ten years. Here then lay the opportunity for pro- 
viding an attractive service which could itself become a 
substantial business. Leasing gained rapid acceptance and 
generated large and fast-growing profits for the lessor. 

These were to some extent dependent upon the rate of 
depreciation, and true profits would be obtained only when 
the lessor had fully recovered all expenses at some time in 
the future, based on the ability to remarket equipment when 
the first user had finished with it. This remarketing capa- 
bility certainly did not exist, nor was it needed, when the 
initial leases were being written. The lessor would certainly 
need this capability and/or other business activities to 
offset the risk inherent in the leasing operations themselves. 

The leasing companies did not have to wait long 
to satisfy these requirements. The size of profit they were 
generating and their rate of growth rapidly captured the 
imagination of the investing public in the USA, providing a 
high multiple for the company's stocks. This in turn gave 
the leasing companies the opportunity for acquisition and 
diversification jYearsley 1973* Oliver 197£J . 
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Lease Contract 



c . 

The lease contract contains characteristics of 
both the purchase and rental contracts. In the computer 
industry., lease contracts are available through "third- 
parties", or directly from the vendors. The third-party 
company will purchase the equipment from the manufacturer and 
lease it to the user. The terms can be flexible and negoti- 
able, depending on the risk to the lessor; thus the longer 
the duration of the lease, the more favorable the terms and 
conditions possible to the user. The lessor must rely on the 
cash inflow (depreciation tax deduction plus cash payments) 
and the residual value of the equipment to cover his costs. 

If the term of the agreement is of relatively short duration, 
the lessor must look forward to the problem of finding a 
second user. 

Lease contracts fall into two general categories: 

1. Full payout or financial leases. 

2. Non-full payout or operating leases. 

In the full payout or financial lease, the user 
(or lessee) essentially has the right3 of purchase and 
assumes the risks normally assumed by the purchaser. The 
legal title, however, is retained by the lessor. The lessee's 
payments are designed to recover for the lessor: 

1. The total cost of the equipment. 

2. The cost of money required to purchase the equipment 
by the lessor. 

3. A contract fee, normally about 0.5^ or more. 
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At termination., the lessor still owns the equip- 
ment, although the lessee will normally have the option to 
purchase. The full payout lease is normally used to obtain 
financial benefit for the lessee; for example, lower payments 
over the useful life of the computer as compared to a rental 
(Szatrowski 1976, Gustafson 1973., Bucci 1973] • 

The non-full payout or the operating lease has 
many characteristics of a rental contract. The essential 
difference is the length of commitment. The term of this 
contract generally starts with a minimum commitment of two 
years, and can go as high as ten. Monthly payments average 
10$ to 30$ less than the manufacturer's rental price. 
Generally speaking, a lease contract (either financial or 
operating) is the most flexible of all contracts abailable 
to a user of computer equipment. The user can negotiate with 
the lessor for terms most beneficial to both parties. These 
negotiations are somewhat unusual since both parties, by and 
large, are aware of each other's financial needs and require- 
ments. Some items that affect the negotiations are: 

1. Maintenance 

2. Depreciation 

3. Investment tax credit 

4. Property taxes and insurance. 

One of the two parties must pay for maintenance, 
and the cost is the same, for either party. There may be 
local advantages for one party or the other to assume the 
maintenance obligation. For example, the user may already 
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have a maintenance contract with the manufacturer for other 
computer equipment and could perhaps extend it to include the 
leased equipment. Alternatively, the lessor may have a 
national contract with the maintenance organization. 

The investment tax credit is a direct tax benefit 
for one of the parties. In certain cases, it could benefit 
one corporation more than another. For example, if one of 
the companies may be operating in a loss period, it may not 
need the investment tax credit since its tax would not be as 
large as in other periods. Another case might occur when a 
company makes massive investments, say an airline in the 
years it purchases new planes; such investments may exhaust 
the potential investment tax credits. In such cases, by 
relinquishing the investment tax credit, the user may be able 
to negotiate a lower lease price. 

There are some additional tax considerations to 
be taken into account in a leasing arrangement. For a trans- 
action to be acceptable as a true lease, i.e., not as an 
installment purchase contract, the lessor is required to 
assume a significant risk both during the lease term and in 
the period after its expiration. According to IRS regula- 
tions the ideal lease arrangement would have characteristics 
among which are these: 

1. Lease payments would be approximately the same through- 
out the basic lease term. 

2. Purchase options are net at fixed amounts but are based 
on fair values at the end of the lease term. 
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3. The estimated fair market value of an asset at the end 
of the lease term is at least 10^ of the asset's original 
cos t . 

4. The lease term is less than 80 % of the asset's useful 
life . 

Also important in the financial analysis of the 
lease contract is the unlimited availability of the equipment 
for the lessee. There are also no overtime use payments 
associated with a lease contract [Szatrowski 1976, Gustafson 
1973, Sabcl 1972, Randolph 1974] . 

2 . Purchasing 

When the user purchases the computer system, he 
acquires ownership of it and can use it one shift or around 
the clock, seven days a week, with minimal increase in hard- 
ware expenses. Maintenance and service of the computer are 
contracted for separately with the vendor. With respect to 
taxes, the user may depreciate the purchased computer system 
as he would any other item of capital equipment. Any opera- 
tion of the system beyond the break-even point constitutes 
pure profit to the user-owner, for he avoids those lease 
payments which he would be making had he decided on an oper- 
ating lease Randolph 1974, Joslin 1977> Graham 197^. 

The break-even point can be calculated as follows: 

The number of months to break even equals the purchase price 
divided by the difference between the monthly lease cost and 
the monthly maintenance cost; (see Figure 8) 
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3E = the number of months to break even 
m 

P_ = the purchase price 

LC m = the monthly lease cost 

M m = the monthly maintenance cost 

The user stands to enjoy certain tax benefits,, but 
he assumes the normal risks associated with ownership: If 

the system fails., the responsibility is his and not the 
manufacturer’s. When he considers purchasing, the user ought 
to take the time value of money into account: Money spent 

today is more costly than the same amount of money spent 
some years from now. Given an interest rate of 10$ per annum, 
one million dollars used to purchase a computer today has the 
same value as 1.6 million dollars spent five years from now. 
a. Purchase Contract 

Under a purchase contract, the purchaser bears 
all the risks of ownership including insurance, taxes, and 
equipment obsolescence. 

By and large, the purchaser will obtain the same 
services and support from the vendor that are available under 
a lease or rental agreement. There are, however, three 
important factors affecting this financial decision: 

1. Full payment must be made to the vendor upon delivery 



of the equipment. 



2. A separate maintenance service contract must be nego- 
tiated since service of the equipment is not considered part 
of the purchase price. 

3. Insurance premiums and appropriate taxes must be paid 
on the asset. 

Assigned values of depreciation can substantially 
affect the cash flow analysis for a purchased system. The 
buyer of any expensive capital equipment should be acquainted 
with the optimum depreciation schedules allowed by law. In 
addition, the future projected tax position in the corpora- 
tion should be considered in order to be able to calculate 
its after-tax cash flow. 

The assignment of a residual (or market) value 
to the equipment at some future date is probably the most 
difficult estimate to make in the financial analysis. If the 
residual value is too optimistic, losses are experienced at 
resale or trade-in time. On the other hand, assigning a 
zero dollar value as residual may be entirely unrealistic. 
Under such circumstances, it may be advisable to assign both 
the most pessimistic and the most optimistic value for 
residual, with analysis under both conditions. Statistically 
it may be possible to determine the most probable outcome 
under these circumstances (Szatrowski 1976, Joslin 1977, 
Randolph 197 ^7 . 

b. Rental Contract 

Under the rental agreement, the user is liable 
for a prepaid fixed minimum payment. The agreement can be 
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terminated by a minimum o? 90 days prior written notice. 

Under this agreement, the risk of ownership remains with the 
vendor. The user has no obligation for such expenses as 
insurance and maintenance; however, he is responsible for 
paying taxes that might be levied on the rental contract by 
the state or local government. 

Extra shift use, over and above the standard 
monthly base hours, represents an additional cost to the 
user. Investment tax credit is also a consideration under 
a rental contract and can be passed to the user. Rental 
contracts find a high level of usage in the computer industry 
due to a number of factors: 

1. Low risk 

2. Financial leverage 

3. Equipment obsolescence 

4. Flexibility 

Flexibility is probably the best argument for a 
rental contract. When the user has a continually varying 
mix of jobs that require different configurations of equip- 
ment, it is to his advantage to be able to move equipment 
rapidly in or out of the installation without penalty charges. 

Straight purchase has two serious drawbacks: 

1. It requires a relatively large sum of money all at one 
period of time. 

2. It dees not permit the activity to adequately test the 
equipment and their system before they have committed them- 
selves to buying something which may turn out to be too large 
or too small. 
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Straight purchase plan should be used only when 
the system life, including reuse, is longer than five years 
and the equipment has had ample time to demonstrate its 
ability to handle a proven application and purchase money is 
available fjSzatrowskl 197 6, Coutinho 1977* Gustafson 1973]. 

In 1977; a revolution was taking place in the 
prices of medium and large-scale computer systems. Prices 
for hardware fell dramatically. Table 12 gives an idea about 
these prices. 

3 . Leasing with Purchase Option 

The main advantage of the lease with purchase option 
is that there is a trial period in which the manufacturer's 
system is tested in the user's applications. If the system 
cannot satisfy the requirements of the applications, it can 
be replaced at relatively little expense to the user. If, 
however, the system fully satisfies the requirements of the 
applications, the user can carry out the purchase by exer- 
cising the option and buying the system. In this case, little 
money will have been wasted since the larger part of the 
total lease payments can be applied to the purchase price. 

Sometimes the purchase option has to be negotiated 
separately with the manufacturer, and sometimes, it is a 
standard part of the lease contract. The intent of this 
option is to give a user the right to purchase the system 
within seme specified period of isime, normally one or two 
years. If the right is exercised, some stated percentage of 
the money paid to the vendor is applied toward the purchase 
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EXAMPLES OF COMPUTER SYSTEMS' PRICES 



TABL£ 12 -a 
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EXAMPLES OF COMPUTER SYSTEMS 1 PRICES 
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price. The option removes a great deal of the risk involved 
in ownership of an untested system. The user has a chance 
to see his applications successfully run before he agrees to 
purchase the system. Lease with purchase option plan should 
be used if funds are available and if the system life, 
including reuse, is longer than five years, and if it is more 
economical and practical than the other ownership methods 
a va ila ble . 

4 . Lease to Ownership 

This plan is new on the computer procurement scene 
and is known by several different names: Special Lease- 

Purchase Plan, Alternate Payment Plan, Installment Furchasing, 
etc. These plans are all essentially the same; in that, 
monthly lease payments are made until some given number 
(generally sixty payments) have been made, or until some 
given amount (the purchase price of the system) has been paid, 
and then title of the computer passes to the lessee. Until 
that time the lessee has no obligation beyond a normal lease 
plan [joslin 1977? Bucci 1973J • 

Lease to ownership plans are not offered by all the 
vendors since they are new. Rather, they are made available 
only upon request. Late in 1969, the Automatic Data Processing 
Equipment Selection Office (ADFESO) of the Department of the 
Navy started requesting that some form of a Lease to Ownership 
Plan be offered as one of the procurement alternatives in 
proposals submitted in response to the request for proposals 
issued by that office. After much ignoring of that request, 
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one vendor finally offered a Lease to Ownership Plan in 
place of offering a discount on the cost of his system. 

After that, several vendors offered similar plans and have 
continued to offer such plans on subsequent bids {joslir. 
1977]. 

In order to make Lease to Ownership Plans an accept- 
able procurement alternative, three major problems have to 
be overcome: 

1. The evaluation technique used in selecting the computer 
system must be broadened to consider value outside the stated 
system life, 

2. The purchase alternative of procurement must be 
re-examined and reconsidered on more than just a cost basis, 

3. The tax situation relating to 'gradual ownership 1 of a 
capital investment must be investigated. 

A company's principle interest in Lease to Ownership 
plans might be due to their recognition of the advantages of 
ownership, coupled to an understanding of the unavailability 
of purchase funds within their present economic situation. 

If purchase funds are not available, then some form of Lease 
to Ownership Plan is essential if the activity ever wishes 
to own the system. 
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B. LEASING VERSUS PURCHASING 



As in the case of most business problems, the purchase- 
or-lease problem should be settled on the basis of a careful 
consideration of many factors; both quantitative and quali- 
tative. These considerations must be balanced against one 
another to come out with the optimal solution. One method 
is to consider the relationship between the cost of purchasing 
and the cost of leasing. 

The quantitative factors are: 

1. Cost involved in purchase versus lease arrangements. 

2. The estimated useful life of the machine. 

3. The desired rate of return on investment (ROI). 

Where applicable, income tax benefits and salvage values 

should be considered. By comparing the cost of purchasing 
with the cost of leasing, the financial advantages of both 
methods may be studied. Quantitative analysis can assist 
management in deciding which method of acquisition to use. 
Methods of quantitative analysis are described in Table 13. 
Other costs and factors, such as insurance, risks of owner- 
ship, resale prices, income taxes, and so on, are disregarded 
since their effects can be easily included in the analysis 
when required. 

1 . Methods For Lease-or-Purchase Decision 
a . Method I 

This method compares the cost of purchasing with 
the cost of leasing 'without considering rate of return. 
Cumulative costs incurred, namely purchase cost ir. the first 
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TABLE 13 



METHODS OF QUALITATIVE 
ANALYSIS 



Purchase basis 



Lease basis 



Purchase cost: $600,000 

Estimated useful life: 6 years 

Maintenance charges: 

First three years: $45,000 

per year 

Last three years: $55,000 

per year 



Rental charges for 
first six years 
( including 
tna intenance 
charges) for the 
same usage as 
would be the 
case if 
purcha sed 

outright $200,000 

per year 



Interest on borrowed 
funds 

(desired rate of 

return) 3^ 
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year and maintenance costs in the first year, second year, 
and so on are used. For the example in Table 14, the break- 
even point would occur just before the fifth year of usage. 
Thereafter, the purchase basis affords favorable financial 
advantages . 

b. Method II 

This approach considers purchase price, mainte- 
nance charges, and rental charges amortized over the useful 
life of the equipment (including interest costs) in equal 
amounts. This approach also, as with Method I, looks forward 
(accumulating technique) in analyzing the effects of pur- 
chasing outright or leasing. However, there are two basic 
differences : 

1. The purchase price, maintenance charges, and rental 
charges amortize in equal amounts in time. 

2. The interest (or rate of return) applies to the unamor- 
tized amounts during the period. 

Both of these factors could be assumed to occur 
monthly or more often, but here the amortization and interest 
computations are assumed to occur annually. Since the rate 
of return would be computed on the book value of the invest- 
ment, the interest (rate of return) for each year on the 
purchase price would be as follows: 

First year: 



$ 600,000 x 3 yo $ 18,000 

500.000 x 3 % 15,000 

400.000 x 3$ 12,000 

300.000 x 3/6 9,000 

200.000 x 3/6 6,000 

100.000 x 3/6 3,000 



Total $63,000 
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COMPARISON OF COST OF PURCHASING 
WITH THE COST OF LEASING 



TABLE 14 
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Lease basis exceeds 

purchase basis -- -- -- $ 10,000 $ 155,000 $ 300,000 



A formula for the above computation can be 
expressed as n t 1 (where n is the number of years), and 
can be derived using the formula for the sum of an arithmetic 
progression and the series, expressed as follows: 

S - n/n + n ~ 1 4. n ~ 2 + n - 3 + + n - (n -l) _ n ± 1 

' n n n n~2 

By substituting six years in the formula for n, 
the interest can be computed as follows: 

° | 1 x 3% x $600,000 = 7/2 x $18,000 = $63,000. 

Similarly, a formula can be derived for computing 

interest on the maintenance charges and the rental charges; 

however, in each of these cases, the annual payments are 

considered individual investments when payment is made. 

Therefore, instead of one simple multiplier of C:_ there 

are series of such amounts. The result is the following 

multiplier: -2— (—2— + — L.. + 1). 

2 2 2 J 

The application of the above multiplier would be as follows: 

—5— (~§~ + — + l) x ¥?o x $ annual maintenance or rental 

charge (or incremental charges 
if there should be a variation 
in annual amounts) 

As an illustration, the maintenance charges of $45,000 under 
the purchasing alternative would result in the following 
interest (or rate of return) on the payments: 

-|2- x 0.03 x $45,000 - $18,225 

This would be for six years. Since there is an increment of 
$10,000 increase beginning in the fourth year, additional 
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interest (rate of return) would be $1,350 computed as follows: 

_9_ x 0.03 x $10,000 = $1,350 

2 

The comparison of the two alternatives, using the above 
approach, ’would result in a decision to purchase the equipment. 

To lease the equipment would mean an increase in 
cash outlay of $298,425 over that of outright ownership of 
the equipment, computed in Table 15. The above computations 
assume that all of the amounts are amortized over the useful 
life of the equipment in equal amounts, except for the 
incremental increase in maintenance charges, 
c. Method III 

This method uses the discounting technique 
(present value method or discounted case flow) frequently 
used in other financial situations. It is illustrated by the 
schedule in Table 16. 

The present value method shows the effect cf 
including interest cost in addition to the other costs as 
shown in Method I. In contrast, the break-even point (indi- 
cating that the purchase method is financially advantageous) 
does not occur until after the equipment has been used for 
more than four years; i.e., sometime in the fifth year of 
usage. If other costs and factors are considered significant 
and they can be expressed in monetary terms. Method III 
permits an easy approach to the determination of the total 
financial advantage. 

Since column (f) in Table 15 is a computation of 
the present value at the time the purchase-or-lease problem 
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TABLE 15 
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PRESENT VALUE METHOD 
Incremental Approach 



TABLE 15 
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-Interest cost of 3 per cent was assumed to accrue during the year using the present 
value formula l/(Hi) n : .97087379; .94259591; .91514166; .88848705; . 86260878 ; 

and .83748426; respectively. Amounts were rounded. Negative values shown in 
parentheses denote financial advantage to the purchasing alternative. 



arises (at the beginning of 1973 )j the financial advantage 
of purchasing the equipment under the present value can be 
calculated as shewn in Table 17. The present value approach 
reduces the rental charges (and maintenance charges) to their 
present value for direct comparison with the purchase price, 
since the purchase price is already stated in terms of its 
present value. 

Another approach using the present value method 
is the incremental approach which sets forth when the payout 
occurs, as shown in Table 18 . 

Table 19 shows present value factors, or conver- 
sion factors to convert future cash flows into present 
values. They are factors that when multiplied by an amount 
to be paid in the future, give the present discounted value 
of these funds. For example, at 6 ^ interest, $1,000 to be 
paid in one year is equivalent today to $943.40; $ 1,000 to be 
paid in two years is equivalent today to $ 890 , etc. If 
instead of money to be paid out, it is money to be received 
(or figured in tax deductions), then again a $48,000 tax 
savings a year from now (at 6 %) is worth $48,000 x 0 . 943 d, 
or $45,283.20 today. To give an idea, Table 20 describes 
the IBM 370/145 computer system configuration. The purchase 
column shows the manufacturer's new purchase price for each 
unit. The prices shown are typical and will vary depending 
on optional equipment and manufacturer's price changes. 
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PRESENT VALUE METHOD 
(Project Approach) 



TABLE 17 
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Mainte- Difference Present Cumulative 

Lease nance Favoring Value Present Present 

Year Costs Costs Purchasing Factor Va lue Va lue 

1 $200,000 $45,000 $155,000 .97087379 $150,485.44 $150,485.44 



TABLE 18 
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PRESENT VALUE FACTORS 



TABLE 19 
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IBM 370/145 COMPUTER SYSTEM 



TABLE 20 
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, 335.00 $25,141.00 $2,358.00 $ 22,783 



Tables 21, 22 , 23 > and 24 give the examples of 
Present Value Cash Flow Analysis for manufacturer's rental 
contract, third party lease (full payout), third party lease 
(non-full payout), and purchase (with zero residual value) 
respectively (assuming an IBM 370/145 configuration at 
$1,131,835.00) [Gustafson 1973* Szatrowski 1976, Fowler and 
Starr 1974, Randolph 1974}. 
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PRESENT VALUE CASH FLOW ANALYSIS FOR 
MANUFACTURER'S RENTAL CONTRACT 
(ASSUMING A 370/145 CONFIGURATION AT $1,131,835) 
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Investment tax credit of 3 1/3$ 



PRESENT VALUE CASH FLOW ANALYSIS FOR 
THIRD PARTY LEASE: FULL PAYOUT 

(ASSUMING A 370/145 CONFIGURATION AT $1,131,835) 



TABLE 22 
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^Investment tax credit of 10$ 



PRESENT VALUE CASH FLOW ANALYSIS FOR 
THIRD PARTY LEASE: NON-FULL PAYOUT 

(ASSUMING A 370/145 CONFIGURATION AT $1,131,835) 
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Investment tax credit of 10$ 



PRESENT VALUE CASH FLOW ANALYSIS FOR 
PURCHASE (WITH ZERO RESIDUAL VALUE) 
(ASSUMING A 370/145 CONFIGURATION AT $1,131,835) 



TABLE 24 
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^Investment tax credit of 10$ 



'/Ill . SOLICITING PROPOSALS 



During specification preparation, thought must be given 
to the problem of soliciting and evaluating proposals. 
Soliciting proposals is no real problem; the problem comes 
in keeping control cf the solicitation. The necessary con- 
tact with the vendors is usually a difficult thing to con- 
trol. One or two vendors usually have managed to become a 
party of the "family". In fact, thoughts about the need for 
a (new) computer system probably were initiated by a vendor. 
Investigation of the system requirements developed by the 
system study team are more likely than not to have uncovered 
several equipment requirements that were unique to the 
inside vendor's systems. It is precisely because of items 
like this that vendor contact must be controlled. 

The system requirements of the company are rarely in 
accord with the system capabilities of any one vendor's 
computer. The purpose of the acquisition is to find the com- 
puter system which most closely fulfills the system require- 
ments (not to make the system requirements echo some given 
vendor's equipment capabilities). 

The objectives of the vendor are not the same as the 
company's objectives; since the company will have to live 
with and pay for the selection it makes, the company's objec- 
tives should be met. The only effective way of assuring that 
the vendor's objectives are net dominating the company's is to 
remove most of the influence by removing the vendor. 



152 



At some period during the system study, vendors known to 
have computer systems which might be able to handle the 
system requirements should be asked to discuss their systems. 
Then all vendors should be "locked out" so that they no 
longer can influence the final system specifications. Up to 
the lock-out, one vendor usually will have exerted the most 
influence. The purpose of the presentations by several 
vendors is to reduce this influence and to demonstrate that 
other vendors' systems have desirable features. 

The lock-out requires that one individual within the 
company be established as the sole point of contact with all 
vendors. This person should not be involved directly with 
the preparation of the system specifications. The lock-out 
should continue for the full period of the selection. The 
remainder of this section will concern itself with the 
problem of controlled solicitation and evaluation of the 
proposals. The review of the system requirements, covered 
earlier, is the basis of the solicitation, but there is more 
to a solicitation than just supplying system requirements. 

The vendors could be asked to bid after being supplied 
with nothing more than the system specifications (require- 
ments), a few statements about necessary vendor support and 
the due dates for the submission of proposals. Such a bid 
request might be sufficient, but it does not ensure effective 
vendor contact. Effective solicitation of proposals includes 
both a good specifications package and good vendor contact. 
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A. REQUEST FOR PROPOSAL (RFP) 

A good RFP package should contain, at the very least, 



1. System requirements 

2. Evaluation criteria 

3. System support 

4. Benchmark data 

3. Bidders' conference dates 

6. Check-in dates 

7. Provision for handling questions 

8. Proposal due dates 

9. Vendors' demonstrations and presentations 

10. Contracting conditions 

11. Award dates 

12. General comments 

SYSTEM REQUIREMENTS . This section should contain the 
system requirements, as defined previously. Any limiting 
conditions that were uncovered during the study should also 
be included. 

EVALUATION CRITERIA. The criteria by which the proposals 
are to be evaluated should be explained to the vendors. This 
includes both the factors to be evaluated and their relative 
values and is beneficial to both the user and the vendor. 

The vendor gains in three important ways: 

1. By knowing the rules of the game the vendor is better 
able to decide whether he wants to participate. The decision 



statements concerning the following twelve 
1977, Thrussell 1976] : 
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to play is an important one to the vendor, for while he 
realizes that the returns are high, he also knows that the 
entry fee is high. Before he can ever hope to be awarded 
the contract, he must prepare and submit a proposal. To 
prepare a proposal, a vendor might incur a cost of from $500 
for a response to a simple hardware specification, to $20,000 
or $30,000 for a proposal in which detailed system study is 
required, to over $100,000 for protracted studies of a multi- 
plicity of systems which might additionally involve larger 
benchmark tests. 

2. Knowing the user's evaluation criteria, the vendor has 
a much better understanding of what type of system must be 
proposed, and proposes accordingly. If the user identifies 
cost as having greater importance than run time, the vendor 
may tailor the proposal to a smaller system with fewer time- 
saving devices. If the user indicates, by value assignment, 
that reliability is important, the vendor can propose a 
system complete with error detection and correction features. 
Moreover, he can develop alternative proposals and determine 
with some degree of accuracy which alternative most nearly 
fits the user's requirements. 

3. The vendor has a chance to comment on criteria, prefer- 
ably early in the game, and identify areas in which conditions 
may not be fair or meaningful. It is always possible that the 
user may be willing to modify these conditions. 

When the user discloses so completely the evaluation 
technique he plans to use, he benefits in the following four 
ways : 
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Those whose 



1. Only the proper vendors are bidding, 
system could not possibly win will see the handwriting on 
the wall and drop out. Overall, fewer systems may be 
proposed, but those proposed should all more nearly fit the 
user's requirements. 

2. The need to evaluate a multitude of proposals should 
be minimal because the vendor would have been encouraged to 
discard inappropriate alternative proposals. The user is 
thus in a position to receive system proposals as nearly 
suited to his wishes as vendors can make them. 

3. The user can now receive free expert advice on the 
stated system requirements and on the evaluation method to 
be used. Most vendor comments on the evaluation method will 
minimize the importance of features and abilities their equip- 
ment does not have and emphasize the value of those their 
equipment has. But also there will be valuable suggestions 

on better evaluation methods or in other meaningful areas 
that were not intended to be evaluated originally. Sugges- 
tions of this kind may help the users to get a system better 
suited to their desires and needs. 

SYSTEM SUPPORT. The kind and extent of vendor support 
necessary for attainment of all system objectives should be 
stated, and may extend beyond maintenance and training needs. 
Programming assistance, special subroutines, and other special 
requirements for which the user has a genuine need, may be 
herein defined as prerequisite conditions. 
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BENCHMARK DATA. The benchmark programs to be used as 
system description and validation should be supplied, along 
with sample data and answers by the prospective user. 

BIDDERS' CONFERENCE DATES. The bidders' conference is 
a formal presentation to the vendor, by the user, of his 
system requirements. This conference should be held a week 
or two after the vendors have received the specifications 
package and have had a chance to review the system require- 
ments and desires. At. the conference, any questions on the 
techniques to be used in evaluating the proposal should be 
discussed and resolved. The date for the conference and a 
general explanation of its purpose should be included in the 
request for proposal (RFP). 

CHECK-IN DATES. Check-in dates are dates, determined by 
the user, on which each vendor should indicate whether or not 
he is still engaged in preparing a proposal; if so, if the 
vendor still has any questions, he should ask them at this 
time . 

PROVISIONS FOR HANDLING QUESTIONS. Since questions will 
arise throughout the proposal period, the user should state 
provisions for handling them, not only at the beginning, but 
at the various stages of the selection. 

PROPOSAL DUE DATES. The proposal due dates should be 
explained, along with a clear description of what will happen 
to late proposals. 

VENDORS' DEMONSTRATIONS AND PRESENTATIONS . Some time 
after the submission of the proposals, the vendors should be 
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afforded the opportunity to tell why they submitted as they 
did. Having acquired benchmarks, the vendors should be 
required to provide demonstrations. 

CONTRACTING CONDITIONS. The vendor should be informed 
that any promises he makes will be written into the contract 
and that the contract will have to be signed by a corporate 
official. 

AWARDING AND DEBRIEFING DATES. Dates should be set for 
the awarding of the contract and debriefing vendors. 

GENERAL COMMENTS. This section should contain any 
instructions on the format of the proposal, such as arrange- 
ment of information with the proposal, number of copies to 
be submitted, and so on. The purpose of this section is to 
make selection easier for the user by keeping things as 
uniform as possible among the several proposals. 

The specifications package is the first official state- 
ment of the details that the vendor will have concerning the 
user's problems. Thus the specifications must be stated 
clearly, questions asked by the user should be meaningful, 
and the evaluation method to be used well defined. The 
specifications package establishes the vendors' first thoughts 
on the system. These thoughts are also important to the user 
because if the vendor is given to believe or suspect that the 
equipment request is of low quality, or that the system (or 
evaluation method) is poor, he may abstain from bidding. The 
vendor who decides not to bid may be withholding the system 
best suited for the task. 
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B. VENDOR CONTACT 



Even with the lock-out policy, it will be necessary for 
the company personnel to come into contact with the vendors 
in several ways. First, there should be some time sec aside 
when the vendors and the company personnel can sit down and 
discuss. the specifications package. This will occur normally 
during a bidder conference. There also is a continuing need 
to answer the vendors' questions as they arise. The vendors 
should be allowed to make some form of presentation after 
submitting their proposals, and they should be required to 
provide any demonstrations called for in the specifications 
package. Finally, there will be the pleasure of telling some 
vendor that he has won and the necessity of telling the other 
vendors why they lost. 

1 . Bidders' Conference and Questions 

A bidders' conference is not always necessary espe- 
cially if the specifications are simple and straightforward. 
However, if a bidders' conference is to be held, it is 
essential that the user be well prepared for it. The best 
preparation is a good set of specifications with meaningful 
benchmarks. The evaluation procedure should accurately 
reflect the user's requirements in system capability. At the 
bidders' conference, the user should have his best people 
available to explain and define the requirements and evaluation 
procedures. The user must consider all questions with an open 
mind. Where there is a possibility that some requirement or 
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procedure might be wrong, che question should be held open 
until a proper answer can be found. Answers "off the top of 
the head" are not sufficient. 

Any questions raised during the conference should be 
answered as quickly as possible. All the questions asked 
should be studied to determine whether they affect only the 
asking vendor or all vendors. If the latter, then the point 
should be clarified for all vendors. 

The bidders' conference will not dispel all the 
vendors' questions; even if it did, others would soon appear. 
The best means of enabling vendors to ask questions is a 
telephone call to the asking vendor, then written documenta- 
tion mailed to all vendors. 

2 . Vendors' Demonstrations and Presentations 

When the specifications package proposals require 
that the vendors demonstrate their computer systems, the dates 
for these demonstrations should be left up to the vendors as 
much as possible. In running a benchmark program or other 
demonstration for validation purposes, the vendor may need to 
obtain the proposed components from several other systems 
being readied for shipment. After deriving the timing informa- 
tion required from these demonstrations, the vendor then must 
decide whether to release the components for shipment as 
planned, and perhaps not be able to reassemble the proposed 
system in time for an official demonstration, or to hold the 
components, thus tying up several systems which otherwise 
could be shipped. 



160 



If the vendor is permitted to demonstrate whenever 
he is ready, he can be spared such a difficult decision. 

The user also benefits by an early demonstration, which 
affords him an earlier check on the vendor. If things do 
not go according to the vendor's plans and the demonstration 
does not demonstrate quite what the vendor said it would, 
the vendor has an opportunity to take these findings into 
consideration when preparing his proposal. One or two weeks 
after the vendors have submitted their proposals, they should 
be given an opportunity to make a formal presentation. 

3 . Contracting and Debriefing 

Normally many statements made in the proposal cannot 
be verified. Any which had a bearing on the winning proposal's 
having won should be written into the contract covering that 
system. Statements of this sort are those dealing with either 
the mandatory conditions or the desirable features requested. 

It should be made clear that the contract will require the 
signature of a corporate official. Without such a signature, 
the contract is no stronger than the position of the signer. 

Salesmen, under a strong emotion, have been known to 
state anything 1 . Corporate officials, when they are signing 
their names, are pledging their corporation's funds. This 
can be reinforced by penalty clauses contained in the contract 
which cover late delivery, failure to deliver, and the like. 
These penalty clauses, as well as the general writing of the 
contract, should be handled by the company's legal staff. 
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Immediately after a formal announcement of vendor selection, 
a debriefing should be arranged to ensure that none of the 
losing vendors is left for long in the position of knowing 
they lost, but not knowing why. 

The debriefing is something that should be handled 
in private. Each vendor should be told exactly why he did 
not receive the contract. If the selection was handled 
openly, the vendor should have little dispute since he was 
aware of how his proposal stacked up. Usually, the major 
point of dispute will be centered on adjustments that were 
made to his proposal. If the vendor was made aware of, and 
agreed to, the adjustments during the validation phase of the 
selection, there should be little he can say. Table 25 gives 
the list of the factors which must be included in any 
contract jjThrussell 1976, Joslin 1977, Sabol 1972, Chorafas 
1967 , Webster and Johnson 1976, Yearsley and Graham 1973]. 
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TABLE 25 



ADP CONTRA CTURAL FACTORS 



Equipment inventory (model number, etc.) 

Terms (rental terms or lease, etc.) 

Performance 

Delivery 

Insta llation 

Support 

Maintenance 

Warranty 

Attachment (of other manufacturer's equipment) 

Relia bili ty 

Training 

Technical specification 
Acceptance 
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IX. COMPUTER PERFORMANCE MEASUREMENT AND EVALUATION 



Data processing can be viewed as a production facility 
which is to satisfy the needs of its users. Typical users 
may be the payroll department staff producing paychecks, 
programmers debugging programs, and engineers solving techni- 
cal problems. The needs of these users are to have their 
jobs processed correctly, on time, and economically. System 
performance evaluation is an attempt to determine how well a 
specific system is meeting or may be expected to meet specific 
processing requirements at specific interfaces. This diffi- 
cult task is more easily carried out as three distinct 
evaluation activities: 

1. The Cost Activity. The objective of this activity is 
to determine the one time and recurring costs from the first 
planning stage through the replacement of the system. 

2. The Judgment Activity. The objective of this activity 
is to evaluate the nonquantifiable factors such as: 

What improvements can the vendor be expected to make in his 
product line during the next five years? 

How will these benefit the company? 

What level of system component maintenance can be expected? 

3. The Performance Evaluation Activity. The objectives of 
this activity are to develop meaningful, quantitative measures 
of how the system may be expected to complete a day's work in 
a day and estimate the unused capability available. 



The main interest of this chapter is only the performance 
evaluation activity, and it will be discussed in more detail 
in the rest of this chapter. Before system evaluation can 
begin three inputs are required: 

1. The description of a specific system to be evaluated. 

2. The specific processing requirements. 

3. The identification of each interface across which the 
system is to be evaluated. 

The first input demands that all the components of the 
system be specified prior to the start of the evaluation 
because the evaluation process is being applied to the entire 
system [ Stimler 1$7^> Rosen 1976j. Every hardware device and 
precisely how that device is to be connected in the system 
must be specified. The operating system, user programs, and 
job scheduling procedures also need to be specified. This 
includes identifying when jobs will be run and which jobs 
are to be multiprogrammed . The accuracy of an evaluation 
depends directly upon how well the system is defined. When 
different configurations are to be evaluated, each must be 
evaluated separately. 

The specific processing requirements must be identified 
for the second input. Typical of the information needed here 
is the work load to be processed and turn around times to be 
met by month, week, day and hour. Periods of heaviest proc- 
essing loads and shortest turn around time requirements are 
of special importance. One processing requirement to eval- 
uate would be the maximum number of transactions per hour the 
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terminal operator can enter and have the computer-generated 
outputs arrive within the required response and turn around 
times . 

The evaluation interface between each user of the system 
and the rest of the system must be identified for the third 
essential input. It is usually a human-system interface, 
such as between a terminal user and the terminal device in 
evaluating a real time system. 

The effectiveness of a system can also be described in 
terms of the capability to process a given workload, and the 
capability to meet time requirements- of individual users. 

The efficiency is measured by internal delays and utiliza- 
tion of individual system components versus demand. Effec- 
tiveness measures are the prime performance measures. Values 
of these measures can be assessed from observations made at 
the external side of the evaluation interface: they are what 

is seen by the system users. These measures are frequently 
called external performance measures [Svobodova 1976] . Effi- 
ciency is an internal factor; values of efficiency measures 
usually must be obtained from within the system. These 
measures aid in identifying problems that diminish system 
effectiveness . 

Examples of both external and internal performance 
measures are given in Table 26 . Performance measures are 
most frequently expressed as mean values. In many cases, 
mean values are clearly inadequate measures of system perform- 
ance. For example, if the variance of the response time of 
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TABLE 26 -a 



EXAMPLES 


OF PERFORMANCE MEASURES 


Performance Measure 


Description 



SYSTEM EFFECTIVENESS 



Throughput 


Amount of useful work completed per 
unit of time with given workload 


Relative 

Throughput 


Elapsed time required to process 
given workload on system Sl/elapsed 
time required to process the same 
workload on system S2 


Capability 

(Capacity) 


Maximum amount of useful work that 
can be performed per unit of time 
with given workload 


Turnaround Time 


Elapsed time between submitting a 
job to a system and receiving the 
output 


Response Time 


Turnaround time of requests and 
transactions in an interactive or 
a real time system 


Availability 


Percentage of time a system is 
available to users 
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TABLE 26-b 



EXAMPLES OF PERFORMANCE MEASURES 



Performance Measure 


Description 



SYSTEM EFFICIENCY 



External Delay Factor 


Job turnaround time/job processing 
time 


Elapsed Time 
Multiprogramming 
Factor (ETMF) 


Turnaround time of a job under 
multiprogramming/turnaround time of 
this job when it is the only job in 
the system 


Gain Factor 


Total system time needed to execute 
a set of jobs under multiprogramming/ 
total system time needed to execute 
the same set sequentially 


CPU Productivity 


Percentage of time a CPU is doing 
useful work (used as a measure of 
throughput) 


Component Overlap 


Percentage of time two or more 
system components operate 
simultaneously 


System Utility 


Weighted sum of utilization of 
system resources 


Overhead 


Percentage of CPU time required by 
the operating system 



Internal Delay Factor Processing time of a job under 





multiprogramming/processing time of 
this job when it is the only job in 
the system 


Reaction Time 


Time between entering the last 
character on a terminal or receiving 
the input in the system and 
receiving first CPU quantum 


Wait Time For I/O 


Elapsed time required to process an 
I/O task 


Wait Time For CPU 


Elapsed time required to process a 
CPU task 


Page Fault Frequency 


Number of page faults per unit of time 
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an interactive system is large, the user is likely to be 
dissatisfied with the system performance even if the mean 
response time is reasonably short. Thus, if the exact dis- 
tribution of the response time is not known, at least the 
variance of the response time ought to be considered as a 
performance measure in addition to the mean response time. 

A good measure of performance of an interactive system is 
the percentile response time. N percentile response time is 
defined as the time limit that guarantees that the response 
times of N percentage of all requests are shorter than this 
limit ^Sekino 1972J . One cannot expect the response time 
for very involved requests to be as short as the response 
time for trivial requests. Thus, response time (percentile 
response time) should be assessed separately for different 
classes of requests. 

Performance measures can be specified only with respect 
to the type and the purpose of the evaluated system, its 
workload, and the purpose of evaluation [^vobodova 197 * 0 * 
Performance measures must be well defined since they set a 
framework for the entire evaluation process. 

Having selected performance measures, the crucial problem 
is to determine how these performance measures depend on the 
system workload and the system structure. An understanding 
of such a relationship is essential if performance optimiza- 
tion efforts are to be constructive, but it is also important 
when selecting a new computer system. An expression of this 
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relationship is the performance model of the system. The 
performance model is the ultimate goal of system analysis 
[Svobodova 1976/ Rosen 1976]. 

The values of performance measures are determined by a 
combination of the following: 

1. Measurement 

2. Analysis 

3. Simulation 

The most accurate values are obtained when the system is 
measured under its real workload. Because of the variability 
or unavailability of the real workload, it is often necessary 
to design an artificial reproducible workload and measure the 
system performance against this artificial workload. When- 
ever evaluating a system that has not yet been implemented 
or is otherwise unavailable for measurement, it is necessary 
to develop a functional model of that system. The values of 
performance measures are then obtained either by analytical 
means or by simulation. 

Measurement and modeling are complementary processes in 
that: 

1. a model provides a framework for measurement, 

2. measurement provides data for validating the model, 

3. the model aids in testing hypothesis and finding 
solutions to performance problems, and 

4. correctness of model predictions is finally verified by 
measurement. 
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A. THE SYSTEM PERFORMANCE EVALUATION METHODOLOGY 

The system performance evaluation methodology requires 
the successful carrying out of the following six steps: 

1 . define the technical terms used, 

2 . establish performance criteria, 

3 . acquire the specific input data needed for each evalu- 
ation, 

4 . analyze the performance of the system being evaluated, 

5 . use appropriate evaluation aids, and 

6 . document the evaluation results. 

Each step will be discussed in detail. 

1 . Define the Technical Terms Used 

To be able to meaningfully answer the question "What 
is the performance of the system?" with "It is operating at 
65 to 85$ of its capability during the third shift", it is 
essential that ooth those asking and those answering the 
questions have a common understanding of all technical terms 
used, such as performance, capability, and system. Since 
there is no industry-wide accepted dictionary of data proc- 
essing terms, different practitioners use the same word to 
have different technical meanings and different words to have 
the same meaning. If meaningful numerical expressions for 
performance are desired, this problem has to be overcome. 

The following guideline could be very satisfactory. 

Start with the assumption that no English word or 
group of words has any inherent technical meaning. Perform- 
ance, system, or time sharing assume technical meaning only 
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after those intending to communicate define each word or 
combination of words they intend to use. Further, the 
essential criteria for the definitions are: 

a. They are clearly understood by those using them. 

b. The definitions should be operational, i.e., permit 
physical measurement to arrive at numerical values. 

c. New definitions should not be originated for commonly 
accepted terms which meet the first two criteria. 

The first criterion of clarity is extremely difficult 
to achieve. The enormity of the task can begin to be appre- 
ciated when it is considered that the single word system is 
being used as a symbol to communicate the idea of a complex, 
operational organization of men, machines, programs, and 
procedures. Therefore, it is suggested that a definition 
which is adequate for effective communication should be 
developed and used at the time it is needed rather than wait 
for a perfect or a standard definition which might be avail- 
able after the immediate need has passed £stimler 197 ^]. 

The operational criteria require that such definitions 
as throughput, response time, and capability permit physical 
measurement to arrive at numerical values when the system is 
operational. The third criterion is intended to reduce the 
proliferation of different definitions for the same concept. 

2 . Establish Performance Criteria 

A performance criterion is a performance standard 
with which comparisons can be made. For example, the cri- 
terion for the throughput of a system is here defined as the 



172 



total data processing work successfully completed during an 
evaluation period. Throughput, like every other criterion 
to be defined, is applicable to all classes of data processing 
systems. However, since the unit of data processing work is 
different for each class, the unit of work unique to the 
class of system being evaluated must be used. 

3 . Acquire The Specific Input Data Needed For Each 
Eva lua tion 

Input data needed include the exact way each compo- 
nent is connected, the configuration and characteristics of 
all hardware and software components, and so on. 

4. Analyze The Performance Of The System Being Evaluated 
A "pencil and paper" analysis is the first essential 

level of analysis. This procedure is: 



a. understand. 


in 


depth. 


the 


opera tion 


of 


the 


system. 


b. understand. 


in 


depth. 


the 


operation 


of 


the 


system 



components , 

c. set up a model of the system, 

d. determine and keep in the model only the significant 
components , 

e. derive mathematical relationships for each of the 
criteria to be used in the evaluation, 

f. insert system characteristics into the mathematical 
relationships and derive the required results, 

g. perform sensitivity tests which indicate the relative 
effect each component has on performance, and 

h. prepare conclusions and recommendations. 
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5 . Use Appropriate Evaluation Aids 



Simulations, benchmarking, and resource utilization 
monitors are available aids for the evaluation process. 

These aids, properly used, can provide cost-effective supple 
ments to the pencil and paper analysis. Improperly used, 
these aids can be expensive, time consuming, and misleading. 

6 . Document The Evaluation Results 

One of the essential results of any performance 
evaluation and performance improvement effort is to document 
what was done, conclusions reached, and recommendations made 
Documentation is an essential step in the methodology [Rosen 
1976, Stimler 197 4? . 

B. THE CONTROL OF SYSTEM PERFORMANCE 

Listing all parameters that affect computer system 
performance would be an exceedingly difficult task. The 
performance of a computer system with respect to a specific 
application is a function of: 

1 . System configuration. 

2. Resource management policies of the operating system. 

3 . Efficiency of system programs. 

4. Effectiveness of the instruction set processor. 

5. Speed of hardware components. 

Performance characteristics are shaped in three stages: 

1 . system design, 

2 . system implementation, and 

3 . matching the system to a given workload. 
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Most of the performance evaluation and optimization 
efforts are presented in stage three, because each informa- 
tion processing system handles a different workload. This 
stage is concerned mainly with system configuration and 
resource management; that is, allocation and scheduling of 
Processor Memory Switch (PMS) components. 

Performance of a particular computer system installation 
can be controlled in several different ways: 

1. Adjustment of system control parameters, 

2. Change or modification of resource management policies, 

3. Balancing the distribution of load among system com- 
ponents through system reconfiguration (changes in the 
assignment of peripheral devices to channels or the assign- 
ment of files to physical storage devices, changes in the 
distribution of software components in the system memory 
hierarchy, etc.), and 

4. Replacement or modification of system components. 

As long as the user interface does not change, the system 
does not change to the user; only the performance does. 

However, configuration changes and software changes result 
in a new system, a system that has to be designed, analyzed, 
implemented, tested, and documented. Control parameters can 
be changed as needed without having to test the system 
operationally. Table 27 lists some system parameters that 
can be used to control system performance. Control para- 
meters can be set either before the system is started, or 
modified if necessary during system operation £svobodova 1976] . 
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TABLE 27 



EXAMPLES 


OF CONTROL PARAMETERS 


Control Parameter 


Description 


Quantum Size 


Time quantum in which the CPU of a 
time-sharing system is allocated to 
jobs 


Internal Priority 


Priority based on the demands of a 
job and services already received 


Degree of 
Multiprogramming 


Number of jobs that are simultaneously 
in the main memory and thus eligible 
to use the CPU 


Memory Partition Size 


Amount of main memory allocated to 
a single job 


Window Size 


Time interval for determining the 
working set of a job 


Maximum Allowed 
Paging Rate 


Maximum allowed paging rate in a 
demand paging system 


Page Survival Index 


Number of CPU bursts received by a 
program before an unreferenced page 
is removed from main memory 


Number of Simultaneous 
Users 


Number of terminal users logged onto 
the system 


Device -to -Channel 
Assignment 


Assignment of I/O devices to avail- 
able channels 
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In the latter case, changes may have to be induced by the 
operator, or control parameters can be changed automatically 
in response to changing user requirements. 

Turnaround time or response time measures not only the 
system performance but also the quality of the program that 
constitutes the job. Performance improvement with respect 
to a specific application ought to be approached from both 
sides: reducing the amount of work required by the applica- 

tion, and improving the efficiency of the system [Ferrari 
1975; Hatfield 1971/ . 

Sometimes improvement of system performance with respect 
to a particular performance measure is possible only at the 
cost of reducing performance with respect to some other 
measures. The qualitative value of a specific level of a 
performance measure is the user's preference for this level. 
Performance trade-offs can be resolved only if the relative 
preferences for different levels of different performance 
measures are known. Determination of the preferred 
conbination is the basic problem of decision theory. 

The system performance can also be assessed in terms of 
the cost of using the system. The cost of using the system 
is a function of the system cost and the cost of the program- 
mer. The higher the throughput, the lower is the system cost 
per unit of work. The shorter the response time, the less 
the programmer's time is wasted waiting for response and the 
lower is the cost of programming. As the system approaches 
its capacity (maximum throughput), the response time suffers. 
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A proper balance between throughput and response time has to 
be established such that the cost of using the system is 
minimized. 

An important factor that influences the productivity of 
a system user is the ease of using the system for a specific 
application. This factor has received more attention under 
the label "human engineering". The response time belongs to 
the category of human-oriented considerations; however, it 
is neither the only important consideration nor the most 
important consideration. Ease of use and performance are 
frequently conflicting design requirements. Since both of 
these factors can make a user's task either satisfying or 
frustrating, there is no simple rule as to how to resolve 
this conflict [svobodova 1976J. 

In general, several different system models are used 
during various stages of a performance evaluation project. 
These models can be divided into three general classes: 

1. Structural models 

2. Functional models 

Functional models, used in performance analysis, can 
be divided into four groups: 

a. Flowchart models 

b. Finite-state models 

c. Parallel nets 

d. Queuing models 

3. Performance models 
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A structural model describes individual system components 
and their connections. Such a model provides a useful inter- 
face between the real system and a more abstract model. 
Structural models are most frequently represented by block 
diagrams. The level of detail in a block diagram can easily 
be varied since individual blocks can in turn be further 
laid down as self-contained block diagrams. Block diagrams 
generally shov; the paths of data flow as well as control 
flow, but they do not specify the conditions governing this 
flow. Thus, block diagrams are suitable 'only as the first 
general level description of the system under study. 

A functional model describes how a system operates. It 
defines the system such that the system can be analyzed 
mathematically or studied empirically. 

Flowchart models are suitable for studying program 
efficiency and execution time requirements. A flowchart 
model is a directed graph model where the nodes represent 
computational tasks and the arcs show the possible flow of 
control between tasks. Flowchart models of system components 
and users’ programs can be used as building elements of a 
system model, tied together by a mechanism that simulates 
system resource allocation and scheduling ^Anderson 1976^ ♦ 

A finite-state model can be used for analysis of utiliza- 
tion of computer system resources. A finite-state model can 
be represented by a directed graph and, in this case, the 
nodes represent the state of the system; the arcs represent 
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the transitions between states. The system state is composed 
of the states of individual system components and it thus 



Parallel nets are directed graphs made of two different 
types of nodes: transitions and places. Places with arcs 

directed into a transition are the conditions that must be 
satisfied concurrently if this transition is to occur. Such 
nets were found to be a useful aid in the design and imple- 
mentation of a simulation model and in a planning of measure- 
ment experiments. 

In a queuing models concept, a computer system is a set 
of resources and queues for these resources. When a job 
enters the system, it is placed in one of the queues where it 
waits until the requested resource becomes available. After 
a request has been processed, a job either leaves the system 
or enters some queue again. Queuing models emphasize the 
flow of jobs through the system, but they also enable one to 
observe the state of the system. These models are the most 
widely used models in computer performance analysis. 

A performance model formulates the dependence of perform- 
ance on the system workload and the system structure. It is 
derived by analysis of a functional model for a specific 
model of workload. 

C. CLASSIFICATION OF DATA PROCESSING SYSTEMS 

Commercial data processing systems can be generally 
classified into three classes: 

1. Batch systems 



relects concurrency of system operations 
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2. Real time systems 



3 . Interactive time sharing systems 

Each class fits the definition of a data processing 
system in that it is an organization of hardware, software, 
user programs, procedures, and people capable of transforming 
specified inputs into specified outputs. However, from both 
the performance evaluation and the system design viewpoints, 
each is sufficiently different to require a separate classi- 
fication. An essential difference among the classes is the 
unit of data processing work. Table 28 shows the character- 
istics used to determine the classification of a system 
{stimler 197^* Rosen 1976, Wooldridge 1973J . 

D. THE UNIT OF DATA PROCESSING WORK 

The unit of work for each class of system is briefly 
described in this section. 

1 . Batch Systems 

The processing of a job, as identified in the job 
logging routine, can be used as the unit of measure for batch 
processing work. Jobs vary widely in the amount of input, 
processing required, storage used, and output generated. 

The characteristics of jobs may also vary during different 
parts of each day. The specific jobs and input data frequently 
vary with day of the week, week of the month, and month of 
the year. For evaluation it is necessary to determine a 
representative workload profile. In many batch processing 
facilities a full month is needed to process an acceptable 
representative work load. In engineering facilities a week 
may be enough. 
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SIMILARITIES AND DIFFERENCES AMONG 
BASIC SYSTEM CLASSES 



TABLE 28 
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Class Characteristic 
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2 . Real Time System 



The processing of a transaction is the unit of work 
for this class of system. The processing of a transaction 
includes the receipt of the input, its processing, and trans- 
mission of all required outputs. Each transaction is com- 
pleted in seconds and there usually are a limited number of 
different transactions a user can input. There may be from 
two to fifteen different transaction types. The combination 
of the limited number of different transactions and the 
short processing time per transaction permits meaningful 
evaluation of real time systems in periods as small as ten 
to fifteen minutes. Inquiry and message are commonly used 
to denote a real time input and output. Transaction is used 
to include these terms. 

3 . Interactive Time-Sharing Systems 

An interactive time-sharing system provides each 
terminal user with essentially all the system capabilities 
he would have at the computer console except that he must 
share the computer resources with other users. This capa- 
bility means that one terminal user can be compiling and 
debugging a new program, another running a program for the 
first time, another building a new data base, and another 
generating complex inquiries of a data base. Some of these 
inputs require responses in seconds, others in minutes. 

Normal batch production jobs frequently are run in the back- 
ground when resources are available. 
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The unit of work is the sum of transactions processed 
plus batch jobs processed. To make meaningful evaluations 
and comparisons it is necessary to use the identical mix of 
jobs for each calculation j^timler 197^> Svobodova 1976, 
Borovits 1973} . Basic definitions of performance criteria 
are presented in Appendix A in the form of simple equations 
to facilitate the calculation of numerical values. 

Appendices B and C of this document provide sample 
checklists for hardware and software respectively. They can 
be applied to computer systems or their major parts (CPU, 
peripheral units) to obtain some criteria for performance 
evaluation and comparison. The data obtained can also be 
used in the applications of the computer selection and 
evaluation methodologies. 
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X. CONCLUSION AND RECOMMENDATIONS 



Today's great emphasis on computer systems is an impor- 
tant reason for management's increased concern about major 
expenditures. Talk of acquiring a new computer system causes 
considerable interest at many levels of management. Manage- 
ment has a major role in each of the three principal phases 
of acquisition: systems analysis and design, selection, and 

installation. 

Management must appoint good people and support them for 
the acquisition effort for the new system. The individuals 
appointed to the acquisition team must be able to communicate 
with management to ascertain management's needs and desires. 
Management must also guide and direct the team. They should 
be informed as to the approaches that might be taken in 
acquiring the computer system. The establishment of realistic 
milestones must be required by the management. 

The responsibilities of the management in the three 
phases of acquisition, mentioned above, can be stated as the 
followings . 

A. In the systems analysis and design phase of acquisition, 
ma nagement : 

1. should discourage pioneering with the new system, 

2. should require systems life forecasting, 

3. should demand system design alternatives, 

4. should require meaningful economic justification. 
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5. should insist on common language programming, 

6. must assure the availability of procurement funds, 

7. must review the system requirements, 

8. should require that system specifications (not 
equipment specifications) be issued to the vendors. 

B. In the selection phase of acquisition, management : 

1. must require competitive specifications. 

2. should encourage the issuance of a presolicitation 
letter to assure that the competitive specifications sought 
in the previous step have been achieved. An advanced copy 

of these specifications should be sent to prospective vendors 
for tneir review. The vehicle for sending the specifications 
to the vendors is a presolicitation letter. 

3. should require the . establishment of a formal 
Selection Plan. 

4. should insist that, for any medium to large scale 
procurement, representative benchmark mixes be used for 
workload representation and validation. 

5. should require a formalized evaluation process for 
system selection. 

6. should require the use of System Life Costing in 
the evaluation process. 

7. should review the complete Solicitation Document 
and Selection Plan before the Solicitation Document is 
released to vendors. 

8. must live by the Selection Plan. 
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C. In the installation phase of acquisition, management: 

1. should require that all activities leading to 
installation be scheduled. 

2. must assure that a formal system acceptance test is 
conducted . 

3. must insist that thorough, complete documentation be 
provided. 

The cost of the proposed system cannot be ignored, there- 
fore when choosing a selection methodology basic elements must 
be observed. These are 

1. the assessment of the value of vendors' offerings to the 
buyer (EVALUATION), and the 

2. validation of the vendors' claims (VALIDATION). 

Implementing sophisticated evaluation and validation tech- 
niques used for selecting large or medium-sized computers can 
easily tie up three or more people for one year or more. The 
cost of their time, travel, computer use, and other expenses 
can easily exceed $50,000. 'That expenditure may be justifiable 
for a $5,000,000 computer system, or even for a $500,000 one, 
but it makes little sense for the buyer to spend $50,000 to 
decide how best to spend another $50,000 for a small computer 
system . 

The buyer must face the fact that it is necessary to invest 
a certain minimal amount just to play the computer selection 
game. He must know how he plans to use the desired computer 
system. Determining the need for a minicomputer may take one 
person six months and cost $10,000. The same task with respect 
to the need for a large computer may take a team of five people 
one year and cost $100,000. 
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In short, advancing technology has drastically reduced 
the cost of computer hardware, and at the same time inflation, 
coupled with the demand for more complex computer applications, 
has increased the cost of the human effort associated with 
computer services, namely, the system study, the selection 
process, programming, and operation. 

If the buyer goes the competitive route because of regu- 
lations or otherwise, he should become concerned with ways to 
minimize the cost of the selection process. Since the cost 
of preparing (writing) the Request for Proposals or Solici- 
tation Document is usually somewhat fixed and minimal (once 
a good document has been found for use as a model), only the 
evaluation and validation processes provide opportunities for 
reducing costs significantly. 

Simplification should not be interpreted to mean that it 
is necessary to use an inferior selection methodology which: 

1. May fail to consider all the desired (but not mandatory) 
items or features, 

2. May not facilitate establishing meaningful and under- 
standable relative values between the desirable items, 

3. Does not permit disclosing the relative value of the 
desired items in the request for proposal (system), 

4. Fails to incorporate systems life costing. 

These failings can be avoided by using the simplified 
version of the Cost-Value or Requirements Costing evaluation 
methodology. The more time spent on establishing the values 
of the desirable features, the better the technique becomes. 
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In view of the fact that benchmarking is the only valida- 
tion process which is really defendable, the cost of bench- 
marking must be cut. The high cost of benchmarking has 
always been at the top of any vendor's list of complaints 
about competitive bidding. The biggest .cost factors to the 
vendors are the cost of debugging the benchmark programs and 
the cost of pulling together and holding a system of the 
type necessary to demonstrate the running of the benchmark 
programs. Benchmarking small systems rarely involves complex 
systems, so the only problem is that the vendors will not 
bid on small systems if too much is expected of them in 
debugging or running the benchmark programs. 

To further simplify the validation process, demonstrating 
the benchmark mix can be required of only the winning vendor. 
This not only reduces the vendors' costs, but also greatly 
reduces the buyer's costs, for demonstrations quickly consume 
man-days and travel dollars. 

The particulars (e.g., government/private sector, expected 
system cost, applications, etc.) of the buyer's situation 
must dictate the means to be used by him for simplifying the 
procedure for selecting a small computer which satisfies 
his needs. 
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APPENDIX A 



PERFORMANCE CRITERIA 



EQ. 1 



Total data processing work successfully 
Throughput = completed during an evaluation period 

EQ. 2 



Throughput 

Throughput rate = Wall clock system time 

(in hrs., min. or sec.) 
to process the throughput 



EQ. 3 

Throughput of a repre- 

Average throughput _ serta tive workload 

rate Wall clock system time 

(in hrs., min. or sec.) 
to process the repre- 
sentative workload 

EQ. 3 for real time systems 

Transactions successfully 
completed in a represen- 
. _ , tative workload 

Average throughput = w a 11 -cloc'k systen time 

(usually in min. or sec.) 
to process that workload 



EQ. 3 for batch systems 



Average throughput 
ra te 



Jobs successfully completed in 

a representative workload 

Wall clock system time ( in hrs . ) 
to process that workload 



EQ. 3 for interactive time-sharing systems 



Average throughput 
ra te 



(Representative short job workload 
+ representative long job workload) 
successfully completed during an 

evaluation period 

Wall clock system hrs. expended 
to process that workload 
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EQ. 3 for real time systems multiprogramming 
background batch jobs 



Average throughput 
ra te 



(Representative transaction work- 
load + representative batch work- 
load) successfully completed 

during an evaluation period 

Wall clock system hrs. expended 
to process that workload 



EQ. 4 

Maximum achievable average throughput rate 
Capability = regardless of the timeliness of outputs 
ever a given time period 



EQ. 5 



Maximum achievable average 

Operational capability - throughput rate while meeting 

timeliness requirements 

EQ. 6 



Relative throughput 
ra te 



Average throughput rate 

for evaluation 1 

Average throughput rate 
for evaluation 2 



EQ. 7 

Relative capability 



Capability for evaluation 1 
Capability for evaluation 2 



EQ. 8 



average 

(Capability - throughput) 100 

Percent unused = — — 

capability Capability 



EQ. 9 



Percent throughput _ 
rate change 

EQ. 10 



Re la tive 

(throughput - l) 100 
ra te 



Turnaround 

time 



Elapsed time in hrs., min. or sec. between 
arrival or first character of first input 
input interface and arrival of last charac 
of final output required by that input at 
output interface 



192 



Ct CD 



EQ. 11 



Elapsed time in sec. between arrival of 
last character of an input transaction 
Response time = at input interface and arrival of first 

character of final output at output 
inter fa ce 

EQ. 12 

Turnaround time of a specific job 
processed in a specific multiprogrammed 

Elapsed time _ environment 

multiplication - Turnaround time of the same j ob 
factor (ETMF) processed as the only job in the 

same system 



EQ. 13 

Equivalent throughput rate of N independent systems 
= Sum of throughput rates of N systems 

= Throughput rate + Throughput rate +. . . + Throughput rate 
of system 1 of system 2 of system N 

EQ. 14 

Equivalent capability of N independent systems 
= Sum of capabilities of N systems 

= Capability of + Capability of + ... + Capability of 
system 1 system 2 system N 
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APPENDIX B 



HARDWARE CHECKLIST 

A. Central Processing Unit (CPU) 

1. Organization (word and/or byte oriented) 

2. Processor storage characteristics: 

Real, buffered or virtual processor storage; core or 
monolithic; amount reserved for firmware; net amount avail- 
able for operating system and problem programs. Amount of 
low-speed storage included, if any. 

3. Complement of registers 

4. Memory cycle time 

5. Average "access to processor storage" time 

6. Number of words or bytes accessed per cycle 

7. Instruction repertoire 

8. Instruction mix timing (average execution time) 

Example: (5-byte unpacked fields) 

a . c = a 4- b 

b. c = a + b 

c . c a a + b 

d. Move a to b 

e. Compare a to b and branch 

Instruction mix should be chosen based on expected use. For 
instance, if a significant amount of floating-point work is 
expected, then these instructions should also be timed. 

If the arithmetic instructions are performed in the 
registers, the loading to and storing from registers should 
be included in the timings. 
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9. Special power unit required 

10. I/O channels 

a. Number of channels by type (selector, multiplexor, 
or block-multiplexor) 

b. Maximum speed of each 

c. Attachable units (or excluded units) 

d. Switching capability of attachable units 

e. Simultaneity of operation between CPU and the 
I/O units, as well as between the I/O units themselves 

f. In-board channel (CPU acts as channel processor) 
or out-board channel (channel processor separate from CPU) 

g. Channel diagram of proposed system 

h. Attachable to another CPU 

11. Integrated controllers 

a. Attachable I/O units 

b. Limitations on which integrated controllers may 
or may not be core resident 

c. Degradation of CPU performance caused by the 
integrated controllers 

12. Timers/clocks 

a. Resolution or precision 

b. Maximum time accumulation 

c. Interrupt triggers 

d. Difficulty in setting 

e. Time of day or interval timers 
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13. Power failure protection 

a. Emergency of f-automa tic shutdown sequence 

b. Power fail safe 

c. Standby or secondary power source 

14. Storage protect capabilities 

a. Number of separate areas protected 

b. Fixed areas or software controllable 

c. Minimum area protectable 

15. Compatibility /emulation features 

a. Machines emulated 

b. Software requirements 

c. Limitations 

16. Expandability 

a. Other features available 

b. Maximum storage and channels 
B. Magnetic Tape Units 

1. Number of units 

2. Number of controllers 

3. Densities supported, single or dual 

4. 7-Track/9-Track 

5 . Operating characteristics: Mounting operation (auto- 

load or manual), tape cartridge required or usable, fixed or 
rotatable dial, stress and wear on tape (number of capstans, 
vacuum column, tension arms) 

6. Continuous or incremental recording 

7. Transfer rate 

8. Start/stop time 
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9. Rewind time 

10. Formula for computing effective speed 

11. Error-checking and correcting capability 

12. Automatic or manual switching (between CPUs, channels, 
controllers ) 

13. Expandability: maximum number of units per control- 

ler, controllers per CPU 

C. Card Read/Punch 

1. Rated speed (reflects maximum speed) 

2. Time to process one card (converted to cards per 
minute, this reflects minimum speed) 

3. Card codes supported 

4. Number of stackers and capacity of each 

5. Number of hoppers and capacity of each 

6. Error-checking capability 

7. Buffered, interlocking or cycle steal 

8. Special features: 51 column, punch-feed-read, mark 

sense, and so on 

9. Capability for sorting, collating, interpreting (card 
print ) 

10. Noise level 

11. Reliability 

12. Controller characteristics and limitations 

D. Printer 

1. Rated speed (for designated character set) 

2. Time to print one line 

3. Number of print positions 
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4. Width of form (maximum and minimum) 

5. Quality of print (single and multiple form) 

6. Character set 

7. Skip speed 

8. Carriage tape specifications 

9. Lines per inch 

10. Noise level 

11. Stacker characteristics 

12. Reliability 

13. Buffered, interlocking or cycle steal 

14. Controller characteristics and limitations 

S. Disk or Drum 

1. Capacity 

2. Transfer rate 

3. Access time (seek and rotational delay) 

4. Removable packs or fixed head storage 

5. Special features (such as rotational position sensing) 

6. Channel restrictions (such as attachable only to 
channel number one, or only device on the channel) 

7. Controller characteristics and limitations 

8. Expandability 

9. Reliability 
F. Operator Console 

1. CRT or printer 

2. Keyboard 

3. Speed 

4. Width of display 
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5. Number of display lines visible to operator 

6. Character set supported 

7. Location relative to CPU and I/O units 

3. Noise level 

9. Reliability 

10. Special paper or stock form 

11. Stacker for paper 

12. Ribbon required-expected life 

G. Paper-Tape Reader/Punch 

1. Speeds (transfer rate, start/stop time) 

2. 7- or 9-channel tape 

3. BCD, EBCDIC or ASCII code 

4. Feed and take-up reels or fanfold 

5. Rewinding required 

6. Checking capability 

7. X-on and X-off required 

8. Compatibility with source or destination of tape 

9. Splicing considerations 

10. Reliability 

H. Telecommunications 

1. Controllers (data adapters) 

a . Number of lines supported 

b. Speed of transmission 

c. Leased line or dial-up 

d. Synchronous or asynchronous 

e. Types of terminals supported 

f. Interchange code supported 
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g. Features supported (such as paper tape, answer- 
back, auto-call, multiple-record transmission, polling) 

h. Buffered 

i. Duplex or half-duplex transmission 

j. Error ccrrection/recovery 

2. Modems - See above and below 

3. Communication facility 

a. Leased or dial-up 

b. Multiplexed line 

c. Duplex or half-duplex transmission 

4. Terminal 

a. Type of display (CRT or hard copy) 

b. Input modes (such as keyboard or tape cassette) 

c. Speed 

d. Width of display 

e. Number of lines visible to operator 

f. Interchange code used 

g. Special paper or stock form 

h. Impact or thermal printer 

i. Multiple copies 

j. Pa per-sta eking facility 

k. Intensity adjustment 

l. Visibility of cursor 

m. Error correction/recovery 

n. Hard-wired or acoustic coupler 

o. On-line or off-line transmission 
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I. Other Equipment 

Many other types of equipment may be available to attach 
to or be used in conjunction with the computer system. Each 
requires various considerations regarding performance, suit- 
ability for the purpose, compatibility with other units, 
reliability, operator interface and physical characteristics. 
Listed below are some of these types of equipment: 

1. Microfilm/microfiche 

2. Plotters/graphics 

3. OCR scanner 

4. Array processor 
3. Audio response 

6. MICR 

7. Manual or automatic switching units 

Many other considerations such as power requirements, air 
conditioning, humidity control, floor space, and so on, apply 
to all the hardware. 
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APPENDIX C- 



SOFTWARE CHECKLIST 



A. Operating System 

1. Resident device(s) 

2. Amount of direct-access storage dedicated to opera- 
ting system and work space required 

3 . Processor storage reserved for operating system 

4. Support for anticipated I/O devices 

5. Extent of multiprogramming capability and limitations 

6 . Proposed method of card I/O and print processing 
(SPOOL) 

7. Preexecution I/O device setup 

8 . Ease of operation 

9 . Acceptability of operator messages 

10. Access methods available 

11. Virtual storage-optional or required 

12. Support of automatic switching between channels 

13. Compatibility or emulation support-capabilities and 
limita tions 

14. Complexity and capability of job-control cards/ 
language 

15. Job-accounting facilities 

16 . Operating system and hardware performance statistics 

17. Telecommunication facilities (Remote Job Entry, direct 
data entry/retrieval, time-sharing, and so on) 

18 . Clarity of error codes/messages 
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19. Data-base management features 

20. Facilities for user program library 

B. Compilers/Assemblers 

1 . Languages supported 

2. Adherence to national standard languages and features 

3. Processor storage required for execution 

4. Work space required on direct-access storage 

5 . Maximum program size allowable (number of source 
s ta tements ) 

6 . Devices not supported 

7. I/O addresses absolute or generic 

8 . Subroutine libraries available 

9. Suitability of languages to meet expected needs 

10. Telecommunication features 

11 . Clarity of diagnostic codes/messages 

C. Sort/Merge 

1. Maximum/minimum file size 

2. Maximum /minimum record size 

3. Fixed/variable record lengths 

4. Blocking 

5 . Number of fields in key-maximum key size 

6 . Devices used/required/supported 

7 . Formulas/tables to compute processor storage and I/O 
storage required 

D. Utility Program 

1. List of utility programs available 

2 . Completeness of list to meet needs 
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E. Performance 



1 . Estimate sort timings 

2. Estimate compile/assemble rate 

3 . Estimate operating system overhead 

4. Estimate processing time of problem programs 

5. Estimate compatibility/emulation performance 

6. Predict total throughput of work load including 
operator functions and multiprogramming performances 

7. Benchmark representative sample to confirm performance 

8 . Use of simulation where advisable 

F. System Preparation Requirements 

1. SY3GEN plan 

2 . On-site or remote 

3. Minimum system required to perform 3YSGEN 

4. Amount of time required 

5 . Degree of testing needed 

6 . Vendor assistance 

7 . Education required 

G. Software Availability /Reliability 

1. How long in use by other installation (or when 
available ) 

2 . Other users' experience 

3. Software maintenance 

a. Normal period between updates 

b. Difficulty to maintain 

c. Availability of vendor assistance 
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4. Quality and completeness of documentation 

5. Computer program patent considerations 
H. Vendor-Supplied Application Programs 

1. Extent of library 

2. Programs required 

3. Programs not required but of potential value 
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